-- Logs begin at Tue 2022-06-21 02:21:19 UTC, end at Tue 2022-06-21 03:31:36 UTC. -- Jun 21 02:54:42.330191 dv-cinder-846842-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 21 02:54:42.339708 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 21 02:54:42 2022] *** Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: nodename: dv-cinder-846842-1-iscsi Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: machine: x86_64 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: clock source: unix Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: pcre jit disabled Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: detected number of CPU cores: 4 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: current working directory: / Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: detected binary path: /usr/bin/uwsgi-core Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: your processes number limit is 47680 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: your memory page size is 4096 bytes Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: detected max file descriptor number: 2048 Jun 21 02:54:42.341726 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: lock engine: pthread robust mutexes Jun 21 02:54:42.342473 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: thunder lock: enabled Jun 21 02:54:42.342473 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 21 02:54:42.342473 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 21 02:54:42.385514 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: Python main interpreter initialized at 0x55d30d6679c0 Jun 21 02:54:42.385514 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: python threads support enabled Jun 21 02:54:42.385514 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: your server socket listen backlog is limited to 100 connections Jun 21 02:54:42.385514 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: your mercy for graceful operations on workers is 90 seconds Jun 21 02:54:42.385860 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: mapped 403077 bytes (393 KB) for 2 cores Jun 21 02:54:42.385929 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: *** Operational MODE: preforking *** Jun 21 02:54:42.386196 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: *** uWSGI is running in multiple interpreter mode *** Jun 21 02:54:42.386196 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: spawned uWSGI master process (pid: 80163) Jun 21 02:54:42.386400 dv-cinder-846842-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 21 02:54:42.387090 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: spawned uWSGI worker 1 (pid: 80169, cores: 1) Jun 21 02:54:42.388337 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: spawned uWSGI worker 2 (pid: 80170, cores: 1) Jun 21 02:54:42.388337 dv-cinder-846842-1-iscsi devstack@keystone.service[80163]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 21 02:54:47.808670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 02:54:47.808670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jun 21 02:54:47.809519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 02:54:47.809519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jun 21 02:54:47.809918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 02:54:47.809918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jun 21 02:54:47.810017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 02:54:47.810017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jun 21 02:54:47.857480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 02:54:47.857674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 02:54:47.857754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 02:54:47.857826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 02:54:47.857886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 02:54:47.857950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 02:54:47.858104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 02:54:47.858247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 02:54:47.858334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 02:54:47.858416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 02:54:47.858489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.858550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.858749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.858872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.858956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.859548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.861956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.862939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.863944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.864987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.865905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.866977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.867939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.868035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.868185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.868317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.868405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 02:54:47.868521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.868653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.868739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.868921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.869850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.870913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.871982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.872995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.873984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.874159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.874240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.874411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.874509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.874643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.875870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.876934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.877878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.878996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.879985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.880957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.881925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.882919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.883859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.884965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.885999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.886924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.887865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.888981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.889896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.890877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.891949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.892935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.893987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.894939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.895865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.896999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.897936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.898931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.899930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.900977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.901923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.902878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.903957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.904987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.905920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.906891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.907927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.908991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.909949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.910934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.911970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.912928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.913861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.914955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.915960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.916949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.917865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.918965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.919985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.920954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.921909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.922853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.923944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.924930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.925943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.926951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.927905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.928909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.929871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.930980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.931954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.932954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.933858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.934984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.935996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.936989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.937938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.938870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.939892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.940821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.941849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.942963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.943916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.944045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.944132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 02:54:47.944285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.944543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 02:54:47.944715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 02:54:47.974883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.975048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.975144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.975228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.975303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.975369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.983814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:47.983993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.012303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.012566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.012670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.012869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.157903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.158366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.209157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.209363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.210181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.210421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.272961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.273963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.274040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.274118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.298442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.298591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.298683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.298851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.298987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.299869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.310946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.311300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.323656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.323925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.324998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.325090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 02:54:48.342360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.342539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 02:54:48.342829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.342956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.343047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 02:54:48.343131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 02:54:48.343340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.343466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.343536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 02:54:48.343599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.343791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.343922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.344032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.344125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.344309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.344596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.354961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.355860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.356123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.364223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.364544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.367522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.367777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.367905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.368872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.369994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.370190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.370317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.370533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.397075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.397834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.398685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.398967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.399635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.399926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.404224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.404645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.404929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.405168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.405559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.405840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.412487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.412735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.415967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.416244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.417233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.417519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.417800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.418084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.418358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.418634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.418903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.419165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.421548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.421831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.422109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.422383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.422636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.422901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.423163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.423426 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.437928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.438207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.442395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.442670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.443022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.446678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.447084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.447342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.450971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.451346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.452103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.455722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.456372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.459437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.459733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.460000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.460125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.460254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.460602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.460892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.463638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.463941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.464243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.464571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.464849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.465142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.480054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.480339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.480632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.480906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.484112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.484402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.484680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.484942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.489164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.490864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.491152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.492849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.494367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.494507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.494624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.494818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.494946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.495223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.495559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.498217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.498546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.498834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.499129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.499429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.509221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.509499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.509786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.510049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.511661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.511964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.512259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.513395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.513674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.513962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.514241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.515819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.516122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.516393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.530164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.533477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.534334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.536854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.537735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.538583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.540227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.540535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.540822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.541044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.542860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.544513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.544792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.545065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.548087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.549715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.552346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.553023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.553313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.553701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.553953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.554077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.557211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.557501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.557887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.558158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.563029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.563321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.563604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.567326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.567609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.567881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.571568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.571849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.572131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.572434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.572704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.573090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.573382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.573654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.573926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.574208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.574480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.574735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.575004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.575276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.575847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 02:54:48.576129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.576404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.576697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.576964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.577346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.577628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.577878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.578146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.578423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.578692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.578951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.579231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.579495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 02:54:48.613855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d30d6679c0 pid: 80169 (default app) Jun 21 02:54:48.617321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f950-51a6-49ba-aa96-ff3cc2d21e49 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:48.617616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f950-51a6-49ba-aa96-ff3cc2d21e49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:48.617899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536f950-51a6-49ba-aa96-ff3cc2d21e49 None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:48.620210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d30d6679c0 pid: 80170 (default app) Jun 21 02:54:48.622142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Tue Jun 21 02:54:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:54:56.412301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:56.412947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:56.412947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:56.417373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:54:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:54:56.424066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:56.424842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:56.424983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:56.497370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] Registered counter for database keystone Jun 21 02:54:56.521824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 02:54:56.530514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 02:54:56.711930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 02:54:56.712257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 02:54:56.767906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:54:56.769571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:54:56.904627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:54:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 02:54:57.065397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:54:57.137558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] Registered counter for database keystone Jun 21 02:54:57.159397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 02:54:57.171077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 02:54:57.299043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707be43-9080-4744-94e6-7e38172c8199 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:54:57.304634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:54:57.305308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:57.305591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:57.305871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:57.315083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:54:57.365994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 02:54:57.366259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c5678-1e3b-49cb-9f16-2800f272574d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:54:57.377138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Jun 21 02:54:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 02:54:57.384037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:54:57.399583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ae1eb-e6dd-42e0-bec1-20165413e903 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:54:57.405286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:54:57.406429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:57.406722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:57.406995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] PATH_INFO: `/v3/projects/eab06a8b789343a7bc8eebc83d2a6637` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:57.411393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] RBAC: Authorizing `identity:get_project(project_id=eab06a8b789343a7bc8eebc83d2a6637)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:54:57.460715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 02:54:57.461305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9db7-2436-460e-b98a-597f05489172 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:54:57.465740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Tue Jun 21 02:54:57 2022] GET /identity/v3/projects/eab06a8b789343a7bc8eebc83d2a6637 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:54:59.457606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40f0a6-3ef9-4d3e-8933-3a2d0ee3b397 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:59.458122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40f0a6-3ef9-4d3e-8933-3a2d0ee3b397 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:59.458122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40f0a6-3ef9-4d3e-8933-3a2d0ee3b397 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:59.464885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:54:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:54:59.474100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:59.474816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:59.474965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:59.501625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:54:59.503161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:54:59.512643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:54:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 02:54:59.538023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40f0a6-3ef9-4d3e-8933-3a2d0ee3b397 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:54:59.556931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40f0a6-3ef9-4d3e-8933-3a2d0ee3b397 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:54:59.562654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:54:59.563302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:59.563590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:59.563870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:59.572445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:54:59.574252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc27c0-2d62-4ab3-9ce9-9f559b340826 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:54:59.583862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Tue Jun 21 02:54:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 02:54:59.589814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:54:59.599519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ea885-99c1-4897-b331-2289181825e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:54:59.605785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:54:59.606669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:54:59.606950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:54:59.607232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:54:59.608905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:54:59.610778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa50e66-84f8-4ed5-9959-c5a43ec58f6c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:54:59.612373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Tue Jun 21 02:54:59 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:01.662986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc53dd-9c89-4a2d-9c87-275f95e8a6f8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:01.663568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc53dd-9c89-4a2d-9c87-275f95e8a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:01.663568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc53dd-9c89-4a2d-9c87-275f95e8a6f8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:01.664974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:01.670615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:01.671063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:01.671146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:01.688346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:01.689554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:01.703871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac8e1-6643-40c6-ac79-5dcdd5aae77f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:01.704264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac8e1-6643-40c6-ac79-5dcdd5aae77f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:01.704640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ac8e1-6643-40c6-ac79-5dcdd5aae77f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:01.706838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:01.714056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:01.714437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:01.714822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:01.763727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 02:55:01.764152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 02:55:01.790277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:01.809178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:01.810807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:01.813626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:01.838812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:01.914895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:01.924942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2584e-550d-49f5-8ba8-6f6c55d003c5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:01.936895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:01.962237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:01.963065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:01.963388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:01.963671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:01.975856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:01.987162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:01.988195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:01.989287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:01.994722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.042300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fed28-4c49-49ac-8cc6-48adb26f1350 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.044626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.078327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.079905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.080852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.081141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.081451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.099603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd410395-d706-4f31-a6d0-17597fe16c6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.135016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.135692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.135989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.136263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.137071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:02.137387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.138826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.151294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:02.153410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.154966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.156892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.257389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:02.262254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8978f8-0be1-480d-a8d2-a7abe7a3a646 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.263317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.266048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.301439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.302330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.302619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.302898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.304126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.305652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.307356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:02 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:02.312738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.327729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.366881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db225c1d-08ee-413e-a729-cbc6f053491d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.420138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.420995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.421278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.421553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.436647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347ae5-df34-478f-8177-8e952c9d844e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.442895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.444401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:55:02.467295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:02.468157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:02.473440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:55:02.473440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:55:02.473440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:55:02.473440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:02.473440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:02.474754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.481427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.482242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.482524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.482798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.483990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.485618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.487420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:02.493714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:02.534218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.579777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b607d1-94fe-44e2-9016-dcac2b10f037 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.580800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:02.601618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c642c2-9471-4fb4-a2be-f0f4cd7c06aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:02.619162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.619845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.620128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.620405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.621169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:02.621474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.622950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e90cf-a70a-4829-b8ee-734491ddfd39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.639920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:02.640651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:02.640946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:02.641231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:02.642085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.domain.id=default, user.id=b424d6dcd2eb40949197b229df3c84e2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:02.642402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:02.644276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce5665-5810-42f0-b195-1575cfd1bc99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:02.645071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:02.652967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Jun 21 02:55:02 2022] GET /identity/v3/role_assignments?user.id=b424d6dcd2eb40949197b229df3c84e2&scope.domain.id=default&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:05.007202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964400b-2369-452e-b4fb-99a7ff11693e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.007737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964400b-2369-452e-b4fb-99a7ff11693e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.007737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964400b-2369-452e-b4fb-99a7ff11693e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.009177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:05.014561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.015366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.015850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.037411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:05.038954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:05.065873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40ef3-bc49-42c8-a046-0a492cc2e4b0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.066486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40ef3-bc49-42c8-a046-0a492cc2e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.066959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40ef3-bc49-42c8-a046-0a492cc2e4b0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.068618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/23] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:05.073576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.074058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.074550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.104922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:05.107343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:05.177996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:05.213268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:05.255004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:05.289338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:05.370007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:05.384929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:05.420148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802ea00-fc00-43d9-9663-cdf829401f21 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:05.463604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:05.466947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.469079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.469725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.490721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:05.493796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:05.507525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce09e1-954c-4035-865f-30b19fa2a775 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-957481e6-30cf-4c4e-aa2b-e22c86812088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:55:05.526940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:05.527843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:05.528681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:55:05.528681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:55:05.528681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:55:05.528681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:05.530661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/26] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:05.532341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9702-aa10-4bfe-8fca-49b151ec5034 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.532857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9702-aa10-4bfe-8fca-49b151ec5034 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.533343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9702-aa10-4bfe-8fca-49b151ec5034 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.539124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/27] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:05.540215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9702-aa10-4bfe-8fca-49b151ec5034 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:05.566780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:05.568487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.569009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.569489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.591099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-062b59d5-4188-487a-82ae-84f79ed4b393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:05.606658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:05.607611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:05.608375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:05.620206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:05.621821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.622360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.622893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.653869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:05.655087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:05.690653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9702-aa10-4bfe-8fca-49b151ec5034 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:05.727355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:05.745957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:05.746993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:05.747466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:05.747935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:05.748996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:05.750960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:05.752714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:05.780062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/29] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:05.781245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:05.781920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:05.787664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:05.821007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:05.924217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5cac1-9543-441f-a369-4b3221c49b7d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:05.996389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc16fe5-8eb4-4f73-965b-726ba7939a06 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.007708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.008404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.008718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.009007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.009805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:06.010090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.011519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.028464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:06.029303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.053680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.054364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.054814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.055373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.056725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.058371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06005179-69ad-4cfc-9a68-42843fa92bc9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.141783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.146994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be746f1-a09a-4964-b4b6-4e3d2371e013 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.185388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.186425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.186941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.187434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.188512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.190188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcd997-a59f-4b46-9bbe-b89e7e99ce42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.226437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/32] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:55:05 2022] POST /identity/v3/projects => generated 301 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:06.228112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754832-56e7-4af3-ae48-28d229c4e353 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.228834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754832-56e7-4af3-ae48-28d229c4e353 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.228961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40754832-56e7-4af3-ae48-28d229c4e353 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.230376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/33] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:06.230930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40754832-56e7-4af3-ae48-28d229c4e353 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.287417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.330294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/34] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/roles => generated 219 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:06.332281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071de460-fac9-4361-b173-9bc5909a34c0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.332653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071de460-fac9-4361-b173-9bc5909a34c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.332939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071de460-fac9-4361-b173-9bc5909a34c0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.334716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/35] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:06.335926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.336214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.336616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.359206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:06.361788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:06.375558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40754832-56e7-4af3-ae48-28d229c4e353 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.439582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.440613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.440992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.441385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.443658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.445374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c114361-54e9-41f3-962c-1beed032979c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.447396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/36] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:06 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:06.448472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.448804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.449090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.471112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:06.473232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:06.475291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/37] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:06.477758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.553471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/38] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:06.554861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.589282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.595431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.605255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53604d7-09c4-48cf-b8af-eb1470b15dd7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.647587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.648441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.648854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.649131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.650591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.652197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d029b7-a0b7-4255-9747-07ea1961425f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.653878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/39] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:06.655261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ae7-a5f2-4aa7-b555-4bbd6ca5ee70 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.655556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ae7-a5f2-4aa7-b555-4bbd6ca5ee70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.655839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92ae7-a5f2-4aa7-b555-4bbd6ca5ee70 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.657103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/40] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:06.657797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92ae7-a5f2-4aa7-b555-4bbd6ca5ee70 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.671792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de7c19-f6d8-49a5-91f2-f50d224d0401 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.772630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92ae7-a5f2-4aa7-b555-4bbd6ca5ee70 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.808765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.809583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.809873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.810155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.811366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.812887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.814555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/41] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:06.817290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.818353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.819386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.819386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.819612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.820428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.821873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a270250-7e38-4793-b582-7c3815f8fd70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:06.837780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/42] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:06.839027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.839316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.839597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.844696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.857615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:06.858839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:06.933303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32eb15-8525-4c54-a870-2cc9beaf39f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:06.937913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:06.938997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:06.977568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:06.981816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:06.982902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:06.983190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:06.983808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] PATH_INFO: `/v3/domains/default/users/b424d6dcd2eb40949197b229df3c84e2/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:06.986509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b424d6dcd2eb40949197b229df3c84e2, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:06.988447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:07.009331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/44] 10.222.0.16 () {58 vars in 1361 bytes} [Tue Jun 21 02:55:06 2022] PUT /identity/v3/domains/default/users/b424d6dcd2eb40949197b229df3c84e2/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:07.010174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:07.072721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c96be-0d50-4c63-be99-3808e3a43b17 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:07.122879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:07.123581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:07.123874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:07.124320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:07.125877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:07.127658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12a305-0393-4f03-abb8-d6b949079ac2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:07.128139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:07.149358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4103-3631-414d-853a-a9515c1aa62b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:07.164339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/45] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:55:06 2022] POST /identity/v3/projects => generated 312 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:07.196788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:07.197493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:07.197789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:07.198063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:07.198841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:07.200322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58fc1e-ccfb-4d3f-83dd-ad49ba98b1f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:07.218811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jun 21 02:55:07 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:09.131088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f36b-2f04-4360-9ac4-0a2f6892e1c6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.131640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f36b-2f04-4360-9ac4-0a2f6892e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.131640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f36b-2f04-4360-9ac4-0a2f6892e1c6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.133091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/47] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:09.138430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.138887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.138977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.157276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:09.158479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:09.206061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67264d-433c-4f68-89ac-37fec5e619af None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.206405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67264d-433c-4f68-89ac-37fec5e619af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.206692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67264d-433c-4f68-89ac-37fec5e619af None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.208103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/48] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:09.213140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.213436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.213726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.230157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/49] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:09.231272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:09.232596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:09.253725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.302530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/50] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:09.326408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.330371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:09.338356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:09.359762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb9395-f02c-4afa-8416-992d81d6a7cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.394294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.395019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.395324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.395598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.405828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:09.413645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:09.414478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:09.415277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:09.420760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.430216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f244cc-7540-435e-a4eb-50f421045ed0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.465072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.465867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.466149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.466423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.467609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:09.469144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:09.470784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/52] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:09.476625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.521529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17388a-3ed3-4b40-a94c-88c893e61529 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.553342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:09.556253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.556958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.557240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.557518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.558275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:09.558581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:09.559993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:09.561468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:09.569757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/53] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:09.575065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.580242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09a4a6-08f9-46f3-8a74-81d49c82c1a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.614737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.615428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.615727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.616014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.616790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:09.618282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:09.647780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/54] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Jun 21 02:55:09 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:09.789580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb8281-f4a0-48c7-bfd0-0cc23077f04f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.826486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.827383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.827675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.827952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.829220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:09.830692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:09.832759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/55] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:09.838565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:09.946585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6eebe-6e34-41c6-8313-f79ae6d9968e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:09.972711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:09.981690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:09.982511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:09.982822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:09.983108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:09.984292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:09.986218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ce02-0fc1-470b-acd4-313d057116ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:09.988143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/56] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:09.993890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:10.055304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:10.094772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af24dff-2cf1-4b8e-a72a-4b50e0560d5a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:10.130607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:10.131282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:10.131566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:10.131826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:10.132697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.domain.id=default, user.id=b424d6dcd2eb40949197b229df3c84e2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:10.133012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:10.134499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7394c-6bf9-42d6-a1df-5a92357c7b4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:10.143192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/57] 10.222.0.16 () {56 vars in 1322 bytes} [Tue Jun 21 02:55:09 2022] GET /identity/v3/role_assignments?user.id=b424d6dcd2eb40949197b229df3c84e2&scope.domain.id=default&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:11.680215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d4d8-7957-45d1-bfc5-ed7deb450530 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:11.680813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d4d8-7957-45d1-bfc5-ed7deb450530 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:11.680813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95d4d8-7957-45d1-bfc5-ed7deb450530 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:11.682995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/58] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:11.691904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:11.692282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:11.692493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:11.711190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:11.712457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:11.785244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/59] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:11.810762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95d4d8-7957-45d1-bfc5-ed7deb450530 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:11.922441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95d4d8-7957-45d1-bfc5-ed7deb450530 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:11.956955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:11.957681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:11.957963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:11.958239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:11.959440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:11.960958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057724e-3c30-4a99-99e5-315c03256d01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:11.962584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/60] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:11.968953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:12.035462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:12.087946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93187b57-f480-42e4-9754-a2f72de94687 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:12.122960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:12.123624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:12.123908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:12.124186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:12.125367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:12.126843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edd5da-ac61-4d76-899d-d8eeec42cbaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:12.163313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/61] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:11 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:15.448071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088ee8-98ed-41ce-b070-4d88035e5096 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.449667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088ee8-98ed-41ce-b070-4d88035e5096 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.450152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3088ee8-98ed-41ce-b070-4d88035e5096 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.452971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/62] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:15.458929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62839-a65a-41bf-87d1-679c2a4ca9db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.460623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62839-a65a-41bf-87d1-679c2a4ca9db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.461128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62839-a65a-41bf-87d1-679c2a4ca9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.501264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1e62839-a65a-41bf-87d1-679c2a4ca9db None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:15.502836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1e62839-a65a-41bf-87d1-679c2a4ca9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:15.617840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/63] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:15.626150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357841d-742d-4ae6-b19b-938657367a67 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.626459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357841d-742d-4ae6-b19b-938657367a67 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.627207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357841d-742d-4ae6-b19b-938657367a67 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.636436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/64] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:15.646924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cacf-96cc-43cc-96eb-7b182134d9c8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.647212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cacf-96cc-43cc-96eb-7b182134d9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.647307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f11d1-1dbb-4cbc-90a4-1d328fb9fd50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.647501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cacf-96cc-43cc-96eb-7b182134d9c8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.647747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f11d1-1dbb-4cbc-90a4-1d328fb9fd50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.648250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f11d1-1dbb-4cbc-90a4-1d328fb9fd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.648968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/65] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:15.650397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cacf-96cc-43cc-96eb-7b182134d9c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:15.671789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fd0f11d1-1dbb-4cbc-90a4-1d328fb9fd50 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:15.673351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd0f11d1-1dbb-4cbc-90a4-1d328fb9fd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:15.717531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:15.743784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:15.764436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/66] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:15.764436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4bec6-e1ab-4ec2-83c4-9f1b35840dbe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.764436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4bec6-e1ab-4ec2-83c4-9f1b35840dbe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.764436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4bec6-e1ab-4ec2-83c4-9f1b35840dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.787252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50a4bec6-e1ab-4ec2-83c4-9f1b35840dbe None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:15.788556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50a4bec6-e1ab-4ec2-83c4-9f1b35840dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:15.793519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cacf-96cc-43cc-96eb-7b182134d9c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:15.831324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:15.832359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.832359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.832950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.843459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4c8bf7c3-3b4c-42a8-a54c-8ada21775ff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:15.851271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:15.853181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:15.854773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:15.857415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/67] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:15.858898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62cffa-1712-4819-8786-05a427e7e1c1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.859198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62cffa-1712-4819-8786-05a427e7e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.859479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62cffa-1712-4819-8786-05a427e7e1c1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.860814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/68] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:15.861624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62cffa-1712-4819-8786-05a427e7e1c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:15.869133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/69] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:15.871048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfe40c-56a8-4a0c-8746-8d1c57410692 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:15.871572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfe40c-56a8-4a0c-8746-8d1c57410692 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:15.872059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfe40c-56a8-4a0c-8746-8d1c57410692 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:15.878901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/70] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:15.878901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfe40c-56a8-4a0c-8746-8d1c57410692 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:15.977470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62cffa-1712-4819-8786-05a427e7e1c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:15.979818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:15.989487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfe40c-56a8-4a0c-8746-8d1c57410692 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.015906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.018988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.019920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.020265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.020540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.027701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.028390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.028701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.028985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.029749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:16.030059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.031470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdd79f-3149-4d92-aeeb-847fa7923c62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.031648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d3722e6a-5361-4a50-bb95-f2ad2740e99e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:16.040306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:16.041990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:16.042569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:16.042645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/72] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:16.043212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.043303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.043370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.043434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.043507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.043573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.062269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:16.063480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:16.064199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:16.065537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:16.138787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/73] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:16.139764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.142531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/74] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:16.143227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.250806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.258336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324b3b5-a54c-444a-ae71-8aadc7c6efd8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.273941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3542-7fab-47d0-a406-a8f579dc3d2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.295416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.295639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.296580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.296868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.297149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.315906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.317603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.319353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/75] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:16.320099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.321472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.322743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.323220 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.323699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.325762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.328304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.330221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/76] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:16.331029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.437987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de9c17-e261-477d-b796-24b16445584a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.443725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e64db7-1d05-41af-a757-aa27f04ab359 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.472509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.473387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.473679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.474024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.476974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.477636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.477935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.478197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.479008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:16.479324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.480769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.484809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:16.489131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/77] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:16.489931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:16.493109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:16.494314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:16.495250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.510312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.543474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.596881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a8a4b-ae20-47c7-aa0c-cf731eb6ff2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.601182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0751a-fbe6-428f-8c82-fd5a6b6d3051 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.633928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.634743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.635028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.635305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.636786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.637461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.638037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.638329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.645944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:16.653718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:16.654760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/79] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:16.656342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.710391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.711881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.737114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.756576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.765664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5172ba-10c1-4596-bc45-8d2e4bf3cc6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.801425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.802104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.802386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.802667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.803388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.804918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.813675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/80] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:16.814526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.830540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/81] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:16 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:16.831364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.935605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f047e-b581-441f-b76a-174a936d2db0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.943130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c43a4-7113-4308-b54c-ce02382d06d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:16.974414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.975321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.975600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.975876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.977150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.978444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:16.978952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.980509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/82] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:16.981413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:16.990855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:16.991556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:16.991846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:16.992128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:16.992933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:16.993244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:16.994651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:16.999450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.008369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/83] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.101579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec21ed3-4bb9-481a-b647-21f787fc3364 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.134453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c0607-604f-44df-b7f2-eba7aa8b9f3a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.142444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.143165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.144256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.147714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.148589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:17.149196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.150698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.164223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/84] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.165423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.174435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.175328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.175624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.175943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.178595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.179348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.181139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/85] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.181920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.248824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.267365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.294141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ad4e9-ec7c-4292-b68d-1bea68f23cd4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.296809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103ece8-53b0-483a-8b14-42f74d7f86cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.331051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.331892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.332391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.332839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.334240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.335731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.337517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/86] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.338363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.340151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.341158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.341653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.342413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.416537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.418375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.448141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a072fdf-cb6d-46ba-ad5d-6b33918d051f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.483610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.495805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.496743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.497036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.497321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.498595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.500476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.501199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.503697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/87] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.505213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.534641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/88] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:17.535595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.653391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856162cb-35ae-4967-b45b-16f78cabe417 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.657411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fb909-ee61-4aa2-83df-37b4ada711b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.690074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.690661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.690952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.691235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.705541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.706509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.706811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.707132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.708938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:17.709730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.711765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.719262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/89] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=de0ee47deb904e17990632bc8067f864 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:17.720742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.741283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.773462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:17.775019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:17.814247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:17.877763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1bba-2088-49b1-9000-ed80374ad304 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:17.892269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/90] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:17.893196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:17.935392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:17.936996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:17.937291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:17.937579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:17.985776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:18.020005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10ef3-9209-4fc9-822b-dfca9b805f2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.038071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.039585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.068601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.069512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.069801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.070352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.071859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.073801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.075865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/91] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:18.076925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:18.109574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:18.207650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/92] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:18.208570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:18.210370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba125a1a-8d1a-4ba2-ba6f-b418209ba67e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.252681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.253575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.255193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.255502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.256794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.258438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.260507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/93] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:18 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:18.261454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:18.281559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:18.387449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a260-4341-4662-bd63-24f983a56faf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.433306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.434458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.435182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.435495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] PATH_INFO: `/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.436747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] RBAC: Authorizing `identity:get_project(project_id=6f6022e6fe9343f5bb23140e12eaaf1e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.438246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.439993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/94] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:18 2022] GET /identity/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:18.440793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:18.494355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130476c-d48b-435f-9f33-5dcebf1667fc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.536861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:18.554952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5071-3fdb-46b1-a58e-e899503c7f8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.593033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.593714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.594005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.594287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.595144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:18.595467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.597100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.603761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/95] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:18 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=692ec53a198846cda9892422cde67a24 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:18.604706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:18.617169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.617866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.618159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.618445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.619373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=b424d6dcd2eb40949197b229df3c84e2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:18.619693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.621281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746b9fa-e547-424b-b148-a87e0cd9847b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.640797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/96] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:18 2022] GET /identity/v3/role_assignments?user.id=b424d6dcd2eb40949197b229df3c84e2&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:18.757873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4bb55-3961-4289-a0c2-27be0df2762c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:18.798026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:18.798650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:18.798938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:18.799226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:18.800101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=6f6022e6fe9343f5bb23140e12eaaf1e, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:18.800439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:18.802737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174acac-b130-47c0-b7b7-70b0c5432a1a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:18.809536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/97] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:18 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=6f6022e6fe9343f5bb23140e12eaaf1e&role.id=de0ee47deb904e17990632bc8067f864 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:19.506400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866bfa4-5ef1-4f72-b88e-4d76ed6bd5f2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:19.506924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866bfa4-5ef1-4f72-b88e-4d76ed6bd5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:19.507029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866bfa4-5ef1-4f72-b88e-4d76ed6bd5f2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:19.508704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/98] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:19.513710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:19.514011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:19.514303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:19.531069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:19.537682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:19.538916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:19.566509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557ba7-35dc-48da-8bc7-352b7580d81c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:19.567053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557ba7-35dc-48da-8bc7-352b7580d81c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:19.567403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42557ba7-35dc-48da-8bc7-352b7580d81c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:19.568936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/99] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:19.574240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:19.574533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:19.574815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:19.608428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:19.610064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:19.624498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/100] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:19.651527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:19.653994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:19.773494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529288f1-d43e-4119-91b8-20f15898c534 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:19.786477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/101] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:19.810693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:19.811855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:19.812284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:19.812722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:19.815931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:19.828711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:19.837263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:19.838232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:19.838994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/102] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:19.839709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:19.845365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:19.956373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638f379-2808-4c56-ae0b-9e2a55455f61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.021998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df324207-88c6-466c-b58e-b097344b8fd3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.025952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.026590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.026883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.027167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.027951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:20.028265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:20.029989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:20.047474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/103] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:20.061682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:20.095020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.095977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.096590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.098477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:20.100363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:20.102473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/104] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:20.108356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:20.156143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:20.170823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:20.247878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60493-615c-4acf-8306-d44f3b5a0d95 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.308202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8c17a-6c20-4072-8dff-33e2c80cbade None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.320862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.321770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.322106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.322386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] PATH_INFO: `/v3/users/b8e7c2876e854fadb6084be75d33c1f9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.350595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b8e7c2876e854fadb6084be75d33c1f9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:20.355670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50e9e2-fc3b-4a8e-8327-276135dc97c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:20.358557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/105] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:20 2022] GET /identity/v3/users/b8e7c2876e854fadb6084be75d33c1f9 => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:20.359840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f621-4738-48fd-a90e-1a6d0429bb7a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.360139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f621-4738-48fd-a90e-1a6d0429bb7a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.360437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f621-4738-48fd-a90e-1a6d0429bb7a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.367211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/106] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:20.370014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f621-4738-48fd-a90e-1a6d0429bb7a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:20.389374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.390064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.390454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.390745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.391484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:20.393003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16375510-a6a5-4f58-a4cd-7b87eac8d55b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:20.446547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/107] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:55:20 2022] POST /identity/v3/projects => generated 302 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:20.447933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.448233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.448551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.483447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:20.485091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:20.588548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f621-4738-48fd-a90e-1a6d0429bb7a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.620693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:20.647725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.648436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.648733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.649018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.650877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/108] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:20.677618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:20.681259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:20.785785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:20.787306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:20.867768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a86fa-8db3-40bf-8ca2-339cafe72d35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:20.934965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:20.936147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:20.936659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:20.937156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:20.960367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:20.971428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:20.972300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:20.973180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:20.974421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/109] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:20.981249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:20.987365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/110] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:20.998377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:21.021305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:21.132262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:21.207727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a0c2-096b-4237-83c9-c3a74be6c52a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.244109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d40be-5708-4e19-965e-69e985bfbf41 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.270941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.272080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.272594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.273188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.274173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:21.274678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.276284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.294822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/111] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:21.300300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:21.345701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.346604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.346919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.347209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.353255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.354801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.356629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/112] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:20 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:21.362973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:21.470828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14052-8a0f-4e20-9adc-cd74e9177c80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.520801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:21.531902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.532860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.533154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.533441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.534744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.537084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.539250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/113] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:21 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:21.544906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:21.635410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:21.677652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d14dca-a396-4ef9-85c1-c7def1b6080a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.679279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83204d-3bab-404f-b862-4aeefef1999c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.719537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.720441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.720737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.721080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.722240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.723985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203dbd-c22c-4df2-9575-f29b0ed21362 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.725789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/114] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:21 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:21.727035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ab70-a8c9-4b9f-ab75-e87769e1e527 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.727332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ab70-a8c9-4b9f-ab75-e87769e1e527 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.727616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ab70-a8c9-4b9f-ab75-e87769e1e527 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.728876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/115] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:21.729947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f78e-f419-4281-a940-1c28941068b5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.730237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f78e-f419-4281-a940-1c28941068b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.730518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2f78e-f419-4281-a940-1c28941068b5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.732254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/116] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:21.732985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2f78e-f419-4281-a940-1c28941068b5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:21.734967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.735866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.736357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.736942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.738120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=b8e7c2876e854fadb6084be75d33c1f9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:21.739428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.741327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928f3c9-6d9e-4720-8b48-cecd0cf28f93 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.747947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/117] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:21 2022] GET /identity/v3/role_assignments?user.id=b8e7c2876e854fadb6084be75d33c1f9&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 259 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:21.756442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6ab63-1bfb-4d9e-b8b6-8797ad71b20b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.757315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6ab63-1bfb-4d9e-b8b6-8797ad71b20b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.757853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6ab63-1bfb-4d9e-b8b6-8797ad71b20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.798385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-45d6ab63-1bfb-4d9e-b8b6-8797ad71b20b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:21.800573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45d6ab63-1bfb-4d9e-b8b6-8797ad71b20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:21.809627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:21.854877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2f78e-f419-4281-a940-1c28941068b5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:21.892651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:21.893681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.893969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.894256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.896657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, user_id=9001fa39b72641f0a1cf1d17e323ecc5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:21.898217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e070ac5-31a8-4df4-98b4-6532e325e45a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:21.919793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/118] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:21 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:21.921213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.925631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.926115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.948227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:21.949671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:21.968398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/119] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:21.970417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cffbf-c2fa-4ece-bbb8-63af602fe126 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.971293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cffbf-c2fa-4ece-bbb8-63af602fe126 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.971753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cffbf-c2fa-4ece-bbb8-63af602fe126 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:21.974132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/120] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:21.980704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:21.981304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:21.982022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.004924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:22.007257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:22.044493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/121] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:22.044493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.127125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/122] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:22.127820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.140743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.143698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.202372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a7c41-feb1-43b0-b149-bd9918da9875 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.238381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.239441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.239875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.240296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.252360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d23df-daeb-48dd-b149-dd30e505cf1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.252883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:22.261306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:22.262423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:22.263406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:22.266506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/123] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:22.267588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.291033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.292123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.292535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.292945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.306874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:22.316042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:22.317140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:22.317999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:22.318706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/124] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:22.319417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.400812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.436437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba98a5-bcbe-49aa-b7ea-3c6e3f479d7e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.437680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c6db-ea58-446f-b872-17d5f7ea6275 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.460925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.474258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.475062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.475360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.475642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.476505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:22.476825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:22.478548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:22.487505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/125] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:22.488610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.502059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.502898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.503287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.503579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.524255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:22.541103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:22.542022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:22.542785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/126] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:22.543417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.595351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715e56-9a19-4212-841b-74a788e9d6b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.630843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.631534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.631823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.632109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.632917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:22.633347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:22.634796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:22.637580 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.646379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/127] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:22.647218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.752230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a99f53-8ffc-4026-b72a-70ebe91b137b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.789274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.790219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.790558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.790851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.791626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:22.791945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:22.793871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:22.803399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:22.804082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.814656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717e024-e8aa-4928-9d60-e6253573ec40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:22.853158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:22.854106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:22.854446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:22.854737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:22.859877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:22.859877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:22.859877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/129] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:22.860495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:22.912173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.973404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:22.979610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade55f31-b6c2-4784-8885-707b4cfd9c5f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.019152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.020188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.020503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.020789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.022909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.024433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.026306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/130] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:23.026900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.111231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085bf28-5109-458d-95ff-38ad293d5176 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.139856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df554c-f90c-4d16-95d6-ba83aaebbc1e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.155634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:23.190858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.191971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.192357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.192598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.194088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.195745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.197848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/131] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:23 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:23.198700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.214084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.215016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.215311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.215600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.216950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.218454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.220458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/132] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:22 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:23.221290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.314130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f8ca-2e01-4152-b044-a7018eb8d3cb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.352725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.353614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.353907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.354191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.355427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.357055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.359184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/133] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:23 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:23.360035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.419451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:23.500501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b9d3f-f9bb-4f27-ac04-ca53eeab5305 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.517401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10d58a-50d9-4b0f-8c81-65c4c0d10386 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.538752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.539640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.539930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.540214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] PATH_INFO: `/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.540584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:23.541591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] RBAC: Authorizing `identity:get_project(project_id=6f6022e6fe9343f5bb23140e12eaaf1e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.543317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.545129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/134] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:23 2022] GET /identity/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:23.546086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.564830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.566032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.566563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.567035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/b424d6dcd2eb40949197b229df3c84e2/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.569353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, user_id=b424d6dcd2eb40949197b229df3c84e2, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.571038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c16ac-f87f-4737-838b-448d8cc81325 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.589515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/135] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:23 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/b424d6dcd2eb40949197b229df3c84e2/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:23.591584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f453a-acfb-4e76-93b9-f18af4eeaa75 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.592085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f453a-acfb-4e76-93b9-f18af4eeaa75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.592589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f453a-acfb-4e76-93b9-f18af4eeaa75 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.594347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/136] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:23.595523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f453a-acfb-4e76-93b9-f18af4eeaa75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:23.673041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ec414-9d3f-403f-8d1d-a9ab797cf421 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.692433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:23.716709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f453a-acfb-4e76-93b9-f18af4eeaa75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:23.735570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.736923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.737630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.740194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/692ec53a198846cda9892422cde67a24` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.743981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, user_id=9001fa39b72641f0a1cf1d17e323ecc5, role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.746608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e270fa-6b88-4e0e-acf9-ee404fd4bc3c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.775357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/137] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:23 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/692ec53a198846cda9892422cde67a24 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:23.777138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.777653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.778168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.789798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:23.791145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:23.792661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:23.793211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] PATH_INFO: `/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:23.800713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f6022e6fe9343f5bb23140e12eaaf1e, user_id=9001fa39b72641f0a1cf1d17e323ecc5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:23.803132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:23.813136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:23.816229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:23.835240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/138] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:23 2022] PUT /identity/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e/users/9001fa39b72641f0a1cf1d17e323ecc5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:23.918547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/139] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:23.955102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:24.037077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:24.119863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2611d-78e9-4ab1-846b-56464f1ff419 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:24.163556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:24.168474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.168474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.168474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.168474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:24.168474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b779f-45f4-4088-8c20-e38c4084a4fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:24.170148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/140] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:24.175653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:24.312502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c1fcd-475b-45ce-8cc6-c0018ebe1da0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:24.320755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:24.360490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:24.361192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.361480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.361761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.362495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:24.364042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:24.379809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7f5-2448-4b65-bacc-041261bb3ffd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.380085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7f5-2448-4b65-bacc-041261bb3ffd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.380307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486c7f5-2448-4b65-bacc-041261bb3ffd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.382557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/141] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:24.387843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.388139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.388439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.400623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/142] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:24 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:24.416822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:24.418070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:24.522947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/143] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:24.570684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:24.704246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8c98-fe59-4ca9-92c2-6ae011bfd259 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:24.751921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:24.753092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.753559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.754343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.769237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d332fdcc-da35-480a-b5ff-c81b38062c28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:24.781877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:24.782790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:24.783557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:24.786187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/144] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:24.793582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:24.805824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749198-6788-4aad-970e-e1759c2c5340 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.806338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749198-6788-4aad-970e-e1759c2c5340 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.806808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a749198-6788-4aad-970e-e1759c2c5340 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.808334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/145] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:24.813435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:24.813936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:24.814495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:24.837729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:24.841735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:24.869255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:24.871568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:24.940732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/146] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:24.962086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6434fe-e2a5-418d-85ef-8175e28eea55 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:24.965547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.016113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.017186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.017722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.018170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.019095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:25.019716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.021889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.033314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/147] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:25.074037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.130486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f6e4-25da-40ef-b0c4-a4055d6cc472 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.216448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd619-a695-452e-b623-98b153d00e34 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.233247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:25.237228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.237956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.238295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.238722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.261592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:25.273682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:25.274604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:25.275364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:25.278016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/148] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:25.283727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.295153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.296370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.296889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.297381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] PATH_INFO: `/v3/users/b8e7c2876e854fadb6084be75d33c1f9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.298842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] RBAC: Authorizing `identity:get_user(user_id=b8e7c2876e854fadb6084be75d33c1f9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.300972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.305233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/149] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:25 2022] GET /identity/v3/users/b8e7c2876e854fadb6084be75d33c1f9 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:25.305793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:25.313555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.455119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98184d80-f061-4b57-9d8f-0d20ac7f056f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.493851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.494879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.495361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.495851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.496845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:25.497461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.499094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.504698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ab8bf-1a11-4c67-a629-dd57c80990d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.509920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/150] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:25.517217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.559022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.559926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.560261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.560570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.563912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.565241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.567255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/151] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:25 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:25.594352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.601485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:25.636099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548ef3-3b9c-4df3-8d49-8626abc9f983 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.664203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:25.673342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.675033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.675516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.676768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.678259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.680096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.682219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/152] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:25 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:25.688288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:25.798342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c0cdd-6a06-4074-8d81-3574da49d7c3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.825299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599a55b-1ec3-4675-a116-be7b6cdd171b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:25.873305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.875440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.876852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:25.877368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.877848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.879223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:25.881152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:25.883579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/b8e7c2876e854fadb6084be75d33c1f9/roles/0777af4ce5c04e0f9e1e457a7b65a891` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:25.889967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, user_id=b8e7c2876e854fadb6084be75d33c1f9, role_id=0777af4ce5c04e0f9e1e457a7b65a891)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.898193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:25.900369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:25.927752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/153] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:25 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/users/b8e7c2876e854fadb6084be75d33c1f9/roles/0777af4ce5c04e0f9e1e457a7b65a891 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:25.990790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:25.992660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:26.111546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/154] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:55:26.118603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:26.277500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:26.281827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f57bbf-9b4d-4206-a9dc-2df6610e276e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:26.319793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:26.320708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:26.320990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:26.321264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:26.322468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:26.323952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed6975-2bc9-4cd2-a42a-fb028d5e50a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:26.325716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/155] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:26 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:26.331971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:26.394218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:26.468158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f54c1-2ac6-4d83-9702-37520743d4b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:26.514101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96211-3ab5-4199-952a-1e20543156af None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:26.515179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96211-3ab5-4199-952a-1e20543156af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:26.516161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe96211-3ab5-4199-952a-1e20543156af None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:26.518246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/156] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:26.521099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:26.522056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:26.522606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:26.524059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:26.524571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:26.525059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:26.525531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:26.526566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:26.528442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:26.530263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:26.551746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/157] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:26 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=de0ee47deb904e17990632bc8067f864 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:26.561839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:26.563349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:26.654371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/158] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:26.694310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:26.803114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdace19f-bffa-4a0f-ac3c-5b40d07e5048 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:26.811831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:26.843095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:26.843911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:26.844195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:26.844575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:26.859426 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:26.871217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:26.872245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:26.873115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/159] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:26.881052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:26.994836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:27.002010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6032dd-a52c-4421-8c49-ad63d0c1c763 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.043899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.044611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.044895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.045233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.046068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:27.046376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:27.047902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:27.057143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/160] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:27.062843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.178649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5720-a58f-45d9-a6d1-2ce64e595e42 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.227942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.228867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.229150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.229423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] PATH_INFO: `/v3/users/b424d6dcd2eb40949197b229df3c84e2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.230656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] RBAC: Authorizing `identity:get_user(user_id=b424d6dcd2eb40949197b229df3c84e2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:27.232140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5b15e-e840-4ff2-adc4-9301261b195a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:27.234000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/161] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/users/b424d6dcd2eb40949197b229df3c84e2 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:27.239847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.242259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55300-256b-47df-a749-df89f6ab8496 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.242553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55300-256b-47df-a749-df89f6ab8496 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.243190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df55300-256b-47df-a749-df89f6ab8496 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.244795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/162] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:27.258354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.258700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.259480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.285721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:27.286952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:27.320799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:27.386825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/163] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:27.433132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf56b73-e6bc-4449-8567-b7292dc36291 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.442624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.472657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.473975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.474270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.474555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.512650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:27.576133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:27.577679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:27.590224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8966-0067-433c-a870-8fbc9b454ef3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.659668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.660669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.660951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.661225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.677288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:27.693067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:27.693985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:27.694756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/164] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:27.700162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.734891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/165] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:27.740809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:27.745483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.869258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:27.892712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dd314-2e12-47c6-b9a3-1d9ed8f1bac6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.929644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d805c-49d6-4826-a6eb-10ba85560607 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:27.951079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.952129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.952645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.953139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.954154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:27.954653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:27.956255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:27.975858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/166] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:27.981515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:27.983909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:27.984841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:27.985131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:27.985414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:27.986633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:27.988167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:27.990699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/167] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:27.996480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:28.116406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:28.128639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e9b09-7f12-4da1-903d-de075aadfd6f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:28.157438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:28.167745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bba5-4657-44a7-b8c6-b7b3e468c23e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:28.208511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:28.208511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.208511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.208511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.208936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=b424d6dcd2eb40949197b229df3c84e2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:28.208936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:28.209352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:28.217816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:28.219015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.219491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.219972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.223449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/168] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/role_assignments?user.id=b424d6dcd2eb40949197b229df3c84e2&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:28.224439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:28.226153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e33fd8-c997-4ab4-b15a-778719c00311 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:28.228462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:27 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:28.234067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:28.346023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63963b-7ecd-4257-894d-ba64f8a3109e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.346680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63963b-7ecd-4257-894d-ba64f8a3109e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.347157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63963b-7ecd-4257-894d-ba64f8a3109e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.349407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/170] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:28.372176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.372822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.373319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.383011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65534e04-fe6a-4dd4-9e21-5046ce24ef8a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:28.407670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:28.409293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:28.442578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:28.443273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.443562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.443906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.548988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:28.550647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:28.556166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/171] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:28.567173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:28.600724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:28.684935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/172] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:28.697181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:28.767169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:28.822674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5caa3-6e88-4e91-9779-289e4f23fcae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:28.828313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:28.839471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bbca8-1c5b-49bf-ac40-474a3d0edf6b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:28.891917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:28.892841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.893132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.893417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] PATH_INFO: `/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.897226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] RBAC: Authorizing `identity:get_project(project_id=6f6022e6fe9343f5bb23140e12eaaf1e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:28.898822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:28.900635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/173] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:28 2022] GET /identity/v3/projects/6f6022e6fe9343f5bb23140e12eaaf1e => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:28.903646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:28.904502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:28.904787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:28.905060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:28.911145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:28.935288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:28.943872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:28.944845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:28.945629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:28.953326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/174] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:28.959381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.072954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a1b7e-e29e-4dc1-9e17-89e4aab4ad47 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.138627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.139314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.139603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.139882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.140779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=6f6022e6fe9343f5bb23140e12eaaf1e, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:29.141099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:29.142591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:29.154973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/175] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:28 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=6f6022e6fe9343f5bb23140e12eaaf1e&role.id=de0ee47deb904e17990632bc8067f864 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:29.210906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c29d9-435c-49cc-bed6-48153131a0af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.263675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.264375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.264760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.265021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.265800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:29.266111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:29.267650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138b179-c9b8-4dd6-bf9f-ebfdb943ceb1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:29.284102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/176] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:29.289543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.327961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.404882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fea8-4201-4059-bccb-14704449ae25 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.405207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fea8-4201-4059-bccb-14704449ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.405447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fea8-4201-4059-bccb-14704449ae25 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.407094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/177] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:29.411828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048ac9b-6191-4684-a7a3-6650f36a6532 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.412116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048ac9b-6191-4684-a7a3-6650f36a6532 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.412395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048ac9b-6191-4684-a7a3-6650f36a6532 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.414794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/178] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:29.415951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.416241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.416547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.427805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.444367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e9710-c518-4b41-b935-3732bab8d828 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.451136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:29.452325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:29.512435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.513352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.513641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.513918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] PATH_INFO: `/v3/users/9001fa39b72641f0a1cf1d17e323ecc5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.516319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] RBAC: Authorizing `identity:get_user(user_id=9001fa39b72641f0a1cf1d17e323ecc5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:29.517858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b5b8f-b771-4e30-bf5e-29f88fca06dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:29.519812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/179] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:29 2022] GET /identity/v3/users/9001fa39b72641f0a1cf1d17e323ecc5 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:29.524525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.524817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.525100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.551591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/180] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:29.551999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:29.553148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.554731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:29.650043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/181] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:29.650974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.680483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.701999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397c4dc-88ad-4b73-b2ae-5c0e8158038a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.734875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.740885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.741560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.741850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.742133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.779662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65eade-011b-498d-a1e1-1d1a9ec6daaa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.816031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.817474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.817761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.817966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:29.818072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.819938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:29.828358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ad043577-36b4-434a-b923-f2b7038de024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:29.836606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:29.837641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:29.838594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:29.842709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/182] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:29.844065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749a8cb-dd0e-4aac-938b-a14dd820f089 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.844343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749a8cb-dd0e-4aac-938b-a14dd820f089 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.844670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749a8cb-dd0e-4aac-938b-a14dd820f089 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:29.848538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/183] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:29.848538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749a8cb-dd0e-4aac-938b-a14dd820f089 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.930744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/184] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:29.930744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:29.936041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.956906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749a8cb-dd0e-4aac-938b-a14dd820f089 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:29.965689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:29.989754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e92161-74da-493c-8650-7629937440b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:29.990575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92161-74da-493c-8650-7629937440b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:29.990857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92161-74da-493c-8650-7629937440b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:29.991128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e92161-74da-493c-8650-7629937440b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.001225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e92161-74da-493c-8650-7629937440b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-34e92161-74da-493c-8650-7629937440b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:30.009492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:30.010427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:30.011256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/185] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:30.011996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.012258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.012598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.029077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:30.030346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:30.037226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7724f7-fc0e-418c-8451-4be8c8c83a97 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.070751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.071482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.071766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.072039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.072819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:30.073122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.074511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45829b-3dd7-4e44-ae13-be583358c846 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.082742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/186] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.084475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9d49-b094-4e3b-969f-8f2c9a4edff2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.084760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9d49-b094-4e3b-969f-8f2c9a4edff2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.085035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d9d49-b094-4e3b-969f-8f2c9a4edff2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.086443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/187] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:30.087090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d9d49-b094-4e3b-969f-8f2c9a4edff2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.099699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/188] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:30.100595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.172317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.185272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.193682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d9d49-b094-4e3b-969f-8f2c9a4edff2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.203192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02136d35-d63d-4ac6-9e5f-be0e252d3aed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.227575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.228476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.228761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.229048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.230230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.231728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.233637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/189] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.237253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.237415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.238143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.238995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.239283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.240050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:30.240364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.241813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bf221-f15c-492a-aa5a-5c3c78067507 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.250522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/190] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.251806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.252094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.252373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.270518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:30.271722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:30.337904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/191] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:30.339050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.345916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6c28c-d1ad-4f48-9e68-3583527fbd43 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.384443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.385352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.385604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.385884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.400763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.402328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.404025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/192] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.404807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.409805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.425655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.444224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893c2d-dc0b-4c0f-848f-7d6dfc0755ef None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.478533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.479329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.479611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.479886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.481056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.482529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.484325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/193] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.484959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.508620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a1905-2f23-4f75-9467-4b32d6834b51 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.543570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.544223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.544545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.544808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.545657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=9001fa39b72641f0a1cf1d17e323ecc5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:30.545969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.547402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.553273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/194] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/role_assignments?user.id=9001fa39b72641f0a1cf1d17e323ecc5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=692ec53a198846cda9892422cde67a24 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.554068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.598500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521f8d-3723-4a5b-824f-58c0a81cf1b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.632053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.640467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.641691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.642121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.642541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.644130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.645895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.652291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/195] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:30.652291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.659806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.678460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd03b0a-d511-47ed-8e02-18fac16bfaa3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.720713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.721982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.722269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.722628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.736996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:30.745028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:30.745952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:30.746784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/196] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:30.747444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.773785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a0a3-2c94-4dda-abdb-7fa7f63e8b6d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.809930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.810587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.810867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.811136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.849656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1a0c-2aa8-424f-b503-e0c544342189 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:30.875709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.880118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.881931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.885703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:30.886385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:30.886670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:30.886945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:30.887673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:30.889154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:30.906029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:30.910975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/197] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:30 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:30.911763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:30.994969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/198] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:30.996300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.028088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e5063-4482-4626-a2c5-7601a8f9d550 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.069186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.070190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.070749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.071034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.118398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.123545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1badd-ddb3-4290-b9ad-213894fc9acc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.161611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.164637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.166342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.177872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.178572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.178860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.179138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.179914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:31.180228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.181793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.190809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/199] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.191887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.288072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/200] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:31.289705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.314355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362739af-0bd9-4786-adcc-d0783b5b07f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.350278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.351314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.351602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.351883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.353451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.354839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.356880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/201] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.357607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.395572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.400255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.403203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d01e2-7493-4d74-9116-3af3ebc63436 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.443016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.444329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.444820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.445273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] PATH_INFO: `/v3/users/b8e7c2876e854fadb6084be75d33c1f9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.446713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=b8e7c2876e854fadb6084be75d33c1f9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.448432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.450401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/202] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/users/b8e7c2876e854fadb6084be75d33c1f9 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.455880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.478417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ade805-3130-4e14-b80d-6239ccbeaa44 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.514967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.515856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.516141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.516449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.517821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.519432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.526662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/203] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.527572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.571000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068486a-7dd7-4497-b1c9-fec832c64cb9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.615120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.616280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.616839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.617331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.618369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=9080523cb5044a03844a26b5844886c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:31.618882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.620732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.626365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.628760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/204] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/role_assignments?user.id=9080523cb5044a03844a26b5844886c1&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.630046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:31.639643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38592bc0-46ab-435d-9df7-ee9f52c8c160 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.688519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.691314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.692355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.694978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.695269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.793172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.794782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.857764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc02073-786a-44e8-8cff-1976c170632b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:31.920788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:31.921732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:31.924030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:31.924480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:31.925751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=9080523cb5044a03844a26b5844886c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:31.927319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:31.929011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:31.943318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/205] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/role_assignments?user.id=9080523cb5044a03844a26b5844886c1&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=692ec53a198846cda9892422cde67a24 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:31.953492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:31.961289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/206] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:31.967792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:32.071573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:32.079667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbf3cf-b62d-4d17-aede-46ddf503013c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:32.122806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:32.123987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:32.124440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:32.124850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:32.127132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:32.129316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924eae0-ea92-4b7f-b09e-9d194675c798 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:32.131634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/207] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:31 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:32.137778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:32.262721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4cb5a-afff-4516-be56-8c67275b8800 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:32.314998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:32.315689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:32.324436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:32.324436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:32.324436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, scope.project.id=efdfe2f79f234724a0bdac8105ff3707, user.id=b8e7c2876e854fadb6084be75d33c1f9) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:32.324436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:32.324436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5016df-2a1a-4d97-babd-7983c28de397 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:32.326123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/208] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:32 2022] GET /identity/v3/role_assignments?user.id=b8e7c2876e854fadb6084be75d33c1f9&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:33.394302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61f373-dc3a-4989-b58d-73c80ed3396b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.395211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61f373-dc3a-4989-b58d-73c80ed3396b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.395465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61f373-dc3a-4989-b58d-73c80ed3396b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.397225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/209] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:33.402660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.403053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.403216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.420645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:33.421775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:33.481082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:33.485136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f950af-2d1a-455f-9ac0-b69bab165d20 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.485136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f950af-2d1a-455f-9ac0-b69bab165d20 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.485136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f950af-2d1a-455f-9ac0-b69bab165d20 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.487267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/210] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:33.492110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.492373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.492695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.506955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/211] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:33.512861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:33.514147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:33.533046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:33.589265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/212] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:33.615342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:33.643958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65428e1-a392-4345-b5ff-1f84e8474833 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:33.682468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:33.683564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.684023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.684515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.696844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:33.705422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:33.706324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:33.707277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:33.709362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/213] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:33.715315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:33.716541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:33.724957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9643-3beb-4d16-b827-8b5b29047e9c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:33.769159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:33.770343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.770730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.771112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.772437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:33.774048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:33.775285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:33.782193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/214] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:33.783252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:33.829001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906c9e0-8c94-46aa-bcf9-49f307a02147 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:33.866376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:33.867352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.867818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.868355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.869348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:33.869950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:33.871554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:33.881878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/215] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:33.887863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:33.893031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8e6ad-d4d9-4c53-adf9-5bda4f4838ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:33.930773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:33.931502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:33.931792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:33.932072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:33.932885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:33.934322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e9e5b-4355-4f2e-b083-151c5799b3d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:33.962955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/216] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:33 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:34.014003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24cf5-f16b-4ffa-9485-8b7a23d87e6c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.031196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.058234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.059546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.060028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.060562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] PATH_INFO: `/v3/users/adb6fc4cad6e4372a5abbf970610f950` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.077102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] RBAC: Authorizing `identity:get_user(user_id=adb6fc4cad6e4372a5abbf970610f950)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:34.079167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:34.081573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/217] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:33 2022] GET /identity/v3/users/adb6fc4cad6e4372a5abbf970610f950 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:34.086127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406c72b-7c89-4768-9f03-c7d5304db678 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.086127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406c72b-7c89-4768-9f03-c7d5304db678 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.086127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406c72b-7c89-4768-9f03-c7d5304db678 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.086127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/218] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:34.088226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.091718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.091883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.092144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.110302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:34.111659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:34.127403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.205286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/219] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:34.217317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77c7e0-10ab-45e1-82df-029a05c20137 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.231067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.254745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.255416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.255705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.255985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.302971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.338182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:34.339696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:34.356058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635fae3-1b80-4b02-b744-c8da07eb2d16 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.403479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.405588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.406324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.406783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.422078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bae5e2dd-4994-4d39-9654-a7d0c2514805 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:34.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:34.433472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:34.434267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:34.436660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/220] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:34.438267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa9680-7436-4e77-8dd3-c21bd2c7f2e3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.438688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa9680-7436-4e77-8dd3-c21bd2c7f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.439096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa9680-7436-4e77-8dd3-c21bd2c7f2e3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.440699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/221] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:34.450770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.451469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.452246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.462295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/222] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:34.463244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.475381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:34.477163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:34.480278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.544860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.555628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/223] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:34.564446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.594049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd91ac-baca-41a7-bdfd-2a046d8cfd17 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.638637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.639367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.639656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.640194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.641030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:34.641346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:34.642777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:34.658072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/224] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:34.658725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.683458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc62435-ffbf-44d7-adba-28a3694f8f35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.719890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.721214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.721723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.722228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.723967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:34.725668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:34.728014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/225] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:34.729171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.738556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.768650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf27371-5901-49b2-964d-c6f377a5b0b8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.800201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.808109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.809629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.810258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.810887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.827453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:34.838468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af168e-61c9-47ca-8124-06c8b18a6873 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:34.840620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:34.842132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:34.843094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:34.846565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/226] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:34.847670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.881252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.882447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:34.882557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:34.882944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:34.884233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:34.885806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:34.887562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/227] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:34.888499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:34.961278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b94d3-1670-4807-83b8-d6c4bbd8b49d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.970064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:34.998418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:34.998617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71498b6a-54c0-475a-9723-01115b2c790e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:34.999597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.000672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.001147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.002966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=adb6fc4cad6e4372a5abbf970610f950) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:35.003490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.005265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.013001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/228] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/role_assignments?user.id=adb6fc4cad6e4372a5abbf970610f950&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=de0ee47deb904e17990632bc8067f864 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:35.014199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:35.036400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.037150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.037437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.037714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.038497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:35.038837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.040272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.049650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/229] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:35.055377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:35.092384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:35.146486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e2bc-c3d1-4b3d-b77a-8b196d2d9487 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:35.183471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b9fb-da46-4b6f-a80b-3b0955966b95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:35.184648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.185642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.186022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.186396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.187722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.189349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.191248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/230] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:35 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:35.197290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:35.228129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.231569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.232692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.232880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.234998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.239220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.248645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/231] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:35 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:35.251130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:35.257484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:35.332894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5215-eb2f-496c-becb-0d5adab151c7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:35.360849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:35.373190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.374517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.375839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.375839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/9080523cb5044a03844a26b5844886c1/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.378510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, user_id=9080523cb5044a03844a26b5844886c1, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.380544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.403767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/232] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:35 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/9080523cb5044a03844a26b5844886c1/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 02:55:35.449233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8640f77-c466-43ba-9e93-1461fc6affd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:35.489878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.491045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.491497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.491941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.493462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.495161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10dab9-0f77-4389-ba1e-9cc32c343cc9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.497103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/233] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:35 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:35.505603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:35.617424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e599923-4377-4bc5-9c58-3c081eda6f28 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:35.666870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:35.668121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:35.668618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:35.669094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/9080523cb5044a03844a26b5844886c1/roles/692ec53a198846cda9892422cde67a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:35.671703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, user_id=9080523cb5044a03844a26b5844886c1, role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:35.673812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31497d76-19f9-410a-9c65-da8f15a0ee04 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:35.695450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/234] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:35 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/9080523cb5044a03844a26b5844886c1/roles/692ec53a198846cda9892422cde67a24 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:36.385194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3a53-9a5a-4174-aede-e1ab8d139fef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.385897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3a53-9a5a-4174-aede-e1ab8d139fef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.386375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3a53-9a5a-4174-aede-e1ab8d139fef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.388234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/235] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:36.393212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b389d-fe58-475f-abfa-74bdf1e6fa50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.393719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b389d-fe58-475f-abfa-74bdf1e6fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.394321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b389d-fe58-475f-abfa-74bdf1e6fa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.415997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1e0b389d-fe58-475f-abfa-74bdf1e6fa50 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:36.417180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e0b389d-fe58-475f-abfa-74bdf1e6fa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:36.472908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:36.515014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/236] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:36.568227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e3a53-9a5a-4174-aede-e1ab8d139fef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:36.619680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cecf72-2288-4910-8cf4-9dc077c9c982 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.620010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cecf72-2288-4910-8cf4-9dc077c9c982 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.620237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cecf72-2288-4910-8cf4-9dc077c9c982 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.621745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/237] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:36.653586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:36.654506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.654506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.654787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.685538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e3a53-9a5a-4174-aede-e1ab8d139fef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:36.687714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:36.688948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:36.720104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:36.721030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.721321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.721604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.722848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:36.724445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:36.730640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/238] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:36.731909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:36.832321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607a92b-5faa-46d5-a7e2-47910ac6bcab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:36.866329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:36.867030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:36.867324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:36.867610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:36.868347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:36.869877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:36.881145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:36.904562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/239] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:36 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:36.917684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/240] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:36.967081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.077854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9aada-c474-4073-8138-ad1315bdfe38 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:37.116328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:37.117605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.118091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.118564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.131014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:37.139419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:37.140317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:37.141157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:37.143641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/241] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:37.150392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.276946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776159da-e548-49ff-9798-8a92a4bac5a6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:37.279201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:37.311487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:37.312173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.312484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.312771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.313545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:37.313863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:37.315281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:37.324393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/242] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:37.330082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.390245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:37.434961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada08f-449a-418f-a66c-56ae06614a70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:37.469292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:37.470189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.470478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.470760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] PATH_INFO: `/v3/users/04246b6923fc46c79c520fac5b3ceea7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.485023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] RBAC: Authorizing `identity:get_user(user_id=04246b6923fc46c79c520fac5b3ceea7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:37.486540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc86125-a252-464e-843f-27987d768e87 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:37.488966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/243] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/users/04246b6923fc46c79c520fac5b3ceea7 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:37.494461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.537399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecbde1-773b-45f1-9d46-967e525ba0a5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.537680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecbde1-773b-45f1-9d46-967e525ba0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.537875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecbde1-773b-45f1-9d46-967e525ba0a5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.539555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/244] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:37.544711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.545008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.545292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.565812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:37.567497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:37.617800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb323a2-006b-4b2e-9682-9cf0fda07c0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:37.658178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/245] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:37.661390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:37.662125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.662386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.662671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.698584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.715903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:37.757655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:37.759274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:37.768370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:37.846798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bde37-8ed9-4761-8bf6-937b85691bfd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:37.906235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:37.907398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:37.907692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:37.907978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:37.920459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/246] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:37.927010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:37.931581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:37.948008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:37.949091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:37.950461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/247] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:37.956470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.040437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.078146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475b4b-369e-4207-a57f-70cd0841d8a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.080749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.090078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f566f85-7966-415c-a22e-df5bb4150b99 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.119032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.120035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.120525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.121059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.122021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:38.122531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.124397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.135351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/248] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:38.139936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.139936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.139936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.139936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.140277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.140277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.141514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.144650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/249] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:37 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:38.148098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.260942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2bbc-f7a4-4e95-898c-89efd303c8f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.282205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81cb92-683d-471e-a0da-f3cfa59ba8ae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.299369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.300924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.301667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.302208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] PATH_INFO: `/v3/users/adb6fc4cad6e4372a5abbf970610f950` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.303875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] RBAC: Authorizing `identity:get_user(user_id=adb6fc4cad6e4372a5abbf970610f950)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.305600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d7a7d-5abb-4d83-b07d-7183399c891c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.307632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/250] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/users/adb6fc4cad6e4372a5abbf970610f950 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:38.313913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751378-8770-4a4d-8bcd-9c1be69a2c75 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.314466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751378-8770-4a4d-8bcd-9c1be69a2c75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.314943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751378-8770-4a4d-8bcd-9c1be69a2c75 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.317601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/251] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:38.318565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751378-8770-4a4d-8bcd-9c1be69a2c75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.321858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.322608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.322886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.323985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.324105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=04246b6923fc46c79c520fac5b3ceea7) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:38.324328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.325847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657246fc-a5c8-4801-8b1c-9cd420aea8df None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.327812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.336551 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/252] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/role_assignments?user.id=04246b6923fc46c79c520fac5b3ceea7&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:38.342004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631c86-eb70-4b31-8fc8-b86128f2f617 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.342334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631c86-eb70-4b31-8fc8-b86128f2f617 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.342569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39631c86-eb70-4b31-8fc8-b86128f2f617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.364466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.370570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39631c86-eb70-4b31-8fc8-b86128f2f617 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:38.371465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39631c86-eb70-4b31-8fc8-b86128f2f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:38.448280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/253] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:38.449292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751378-8770-4a4d-8bcd-9c1be69a2c75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.449852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a495-ee95-4cc0-b152-5a33070c649f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.450156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a495-ee95-4cc0-b152-5a33070c649f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.450434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6a495-ee95-4cc0-b152-5a33070c649f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.453781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/254] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:38.458300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.458671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.459026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.479045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:38.480328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:38.491669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.492915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.493429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.493929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.495434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.497290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.499429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/255] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:38.500818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.560981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/256] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:38.562436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.629782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41da33-cabb-49fe-806f-79ba1f94914d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.631650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.635562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:38.671549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.673118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.673610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.674059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.683169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553224a-e2c3-41e5-96bd-373b0e02b4ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.686249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:38.695077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:38.695999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:38.696845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:38.699866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/257] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:38.700876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.720807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.721873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.722319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.722646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/adb6fc4cad6e4372a5abbf970610f950/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.724926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, user_id=adb6fc4cad6e4372a5abbf970610f950, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:38.726418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:38.744666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/258] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:38 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/adb6fc4cad6e4372a5abbf970610f950/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:38.751255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.829841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84ee59-d451-47fb-9f9a-ec08069ca36d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:38.867193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:38.868019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:38.868305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:38.868655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:38.882458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:38.891028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:38.892071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:38.892900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/259] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:38.905693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:38.957042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.034677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8872a-7793-421a-b84d-91af355466c3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.075837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.110194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.110887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.111172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.111448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.112215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:39.112617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.114251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.115397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90955f8-4e3e-444e-888e-88874715b408 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.138030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/260] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:39.145789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.184117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.188934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.189222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.189501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.190265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:39.190575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.192207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.210031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/261] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:39.216039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.292729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7f517-b455-4173-a2d2-851f62999d9e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.335564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.336496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.336789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.337074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.341147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.342655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.344759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/262] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:39.352469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.375607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.410879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2f0d-77e4-4e5d-8d09-5d4da0071502 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.419958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.474825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b3312-2897-48c5-9228-0758b701a918 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.493211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.494128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.494487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.494774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] PATH_INFO: `/v3/users/9080523cb5044a03844a26b5844886c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.496136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] RBAC: Authorizing `identity:get_user(user_id=9080523cb5044a03844a26b5844886c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.497651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.499699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/263] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/users/9080523cb5044a03844a26b5844886c1 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:39.505569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.522617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.523553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.524009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.524438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.669834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.673832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.717834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee43f1-6da8-4bcb-b9c8-672d57e32580 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.751730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.806475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.806721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/264] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:55:39.811172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.811172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.811172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.812464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.865453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:39.914745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3adf3d-be13-4d1f-a34c-b7e824d20dc7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:39.951088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:39.951958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:39.952639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:39.952918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:39.954148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.955633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.958031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/265] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:39.964110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:39.985521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:39.987022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:39.989728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:40.068769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7782f0c-a429-46ef-847c-3eb94b540077 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:40.103920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:40.104670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.104953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.105232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.106079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=9080523cb5044a03844a26b5844886c1) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:40.106394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:40.107877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:40.114747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/266] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/role_assignments?user.id=9080523cb5044a03844a26b5844886c1&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:40.224999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/267] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:40.230312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:40.338464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:40.358356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b5bf-92ff-403d-a90f-74ed4051c5c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:40.397063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:40.398013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.398309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.398696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.399927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:40.401488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4a849-ebbc-4ca1-816a-4bf080036f92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:40.403609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/268] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:40 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:40.409169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:40.466294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:40.524926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b2ea3-c8cc-43de-b84c-3cef6667be34 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:40.572999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:40.573664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.573947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.574227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.575068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=9080523cb5044a03844a26b5844886c1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:40.575450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:40.576979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3372ca-3d6d-4a98-b57b-edd04431ba20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:40.583392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/269] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:40 2022] GET /identity/v3/role_assignments?user.id=9080523cb5044a03844a26b5844886c1&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=692ec53a198846cda9892422cde67a24 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:40.859167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802c44-f11e-4215-90a3-1b7a2e7058e2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.859665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802c44-f11e-4215-90a3-1b7a2e7058e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.859665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2802c44-f11e-4215-90a3-1b7a2e7058e2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.861306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/270] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:40.868922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.869807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.870283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.888917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:40.890525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:40.894916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ce13-c5a9-4318-8d65-28910ecf914a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.895298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ce13-c5a9-4318-8d65-28910ecf914a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.895473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ce13-c5a9-4318-8d65-28910ecf914a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.896886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/271] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:40.903488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:40.903883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:40.904170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:40.928528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:40.929848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:40.978987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/272] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:41.002371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/273] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:41.007437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.026683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.062033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.112126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.124769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639607-6079-42d3-b89e-8085b4aa3639 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.135968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39cde0-3d94-4e5b-94e8-454b16b5e174 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.173730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.173730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.173947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.174195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.182086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.182904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.183184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.183458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.187303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:41.188817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:41.190715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/274] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:41.197302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:41.197473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:41.208888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:41.209825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:41.210626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:41.214483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/275] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:41.220397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.330148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a15f8-af3b-4682-9f88-917870c130e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.343566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5cfbd-f6f1-4a08-998e-c60dff87f2fe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.345949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.370206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.370806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.371089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.371572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.372079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:41.373579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:41.393056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.394292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.395116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.395648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.396779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:41.397295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:41.398907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e68a0-e869-4ebe-961d-8686dfa00aec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:41.404547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/276] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:55:41 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:41.414204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/277] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:41.414393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.421751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.442162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739ec7a-7b32-422c-ae73-abe0a97d5009 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.442506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739ec7a-7b32-422c-ae73-abe0a97d5009 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.442788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739ec7a-7b32-422c-ae73-abe0a97d5009 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.444236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/278] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:41.448523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6eef8-33dc-40f1-b9e1-5e7438033aec None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.448992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6eef8-33dc-40f1-b9e1-5e7438033aec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.449094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6eef8-33dc-40f1-b9e1-5e7438033aec None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.450491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/279] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:41.451896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd63-4001-446e-9c6e-c03b1735dfa8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.452301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd63-4001-446e-9c6e-c03b1735dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.452606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25acd63-4001-446e-9c6e-c03b1735dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.470669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f25acd63-4001-446e-9c6e-c03b1735dfa8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:41.472055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f25acd63-4001-446e-9c6e-c03b1735dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:41.541483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51a29e-ee62-42bc-8eff-c06fabe08578 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.550461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/280] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:41.558692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.559056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.559463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.580335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:41.582137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:41.585125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.586116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.586411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.586687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] PATH_INFO: `/v3/users/860b1ac67ec9407d8a75a1f14edda391` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.603360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] RBAC: Authorizing `identity:get_user(user_id=860b1ac67ec9407d8a75a1f14edda391)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:41.604891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:41.606941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/281] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/users/860b1ac67ec9407d8a75a1f14edda391 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:41.607762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.650593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.663103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/282] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:41.664004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.740803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.743781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d65da-a7b2-4f6f-a4c6-e6e042cfa97f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.786118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca6b73-338a-4e0f-a63f-42f0fbb9f7a8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.786714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.787901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.789218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.789695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.803358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:41.811263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:41.812160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/283] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:41.813559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.836765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:41.838223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:41.838733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:41.839195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:41.840793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, user.id=860b1ac67ec9407d8a75a1f14edda391, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:41.841335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:41.843511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:41.851824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/284] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/role_assignments?user.id=860b1ac67ec9407d8a75a1f14edda391&scope.system=all&role.id=de0ee47deb904e17990632bc8067f864 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:41.852969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:41.970252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:41.981585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c636c-7f76-4cfb-b2ca-9bc6a49a9482 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:41.986154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2e54-a0be-42ea-a3c8-aa646b407c5e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.015412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:42.027902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.029029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.029419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.029829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.030615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.031365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.031647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.031921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.033033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:42.033353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.035304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.044569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:42.046348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/285] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:42.051773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:42.053831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:42.054719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:42.055516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:42.058171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/286] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:42.066065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.162809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67deefde-ef31-45ac-b4ab-c7d3a336850a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.202061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.203319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.203797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.204305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] PATH_INFO: `/v3/users/adb6fc4cad6e4372a5abbf970610f950` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.205747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] RBAC: Authorizing `identity:get_user(user_id=adb6fc4cad6e4372a5abbf970610f950)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.207492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.210272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/287] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/users/adb6fc4cad6e4372a5abbf970610f950 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:42.217330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.228569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090958-14c1-49cf-8104-0a0de3a6a267 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.261835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.262514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.262804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.263075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.263830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:42.264144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.265762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.280068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/288] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:42.282139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.316892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:42.347236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:42.408525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a40431-7bfa-411b-ab00-6cdde190a052 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.445414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.446491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.446873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.447247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] PATH_INFO: `/v3/users/04246b6923fc46c79c520fac5b3ceea7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.448611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] RBAC: Authorizing `identity:get_user(user_id=04246b6923fc46c79c520fac5b3ceea7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.450312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.456671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/289] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/users/04246b6923fc46c79c520fac5b3ceea7 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:42.458258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.540662 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:42.542341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930bc37-85cf-430c-9cfc-4652238333ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.566190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10117961-7c5c-4d0f-84b5-6968118a31e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.612872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.613756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.614041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.614325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.615961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.618862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.622627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.623292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.623576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.623853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.631752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/290] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:42.637700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.735201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.736838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.796479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:42.845571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/291] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:42.856298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:42.863322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0bef8-6520-43d0-aff4-1b24ad1227a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:42.900033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:42.901566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:42.901854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:42.902128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/04246b6923fc46c79c520fac5b3ceea7/roles/0777af4ce5c04e0f9e1e457a7b65a891` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:42.904404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, user_id=04246b6923fc46c79c520fac5b3ceea7, role_id=0777af4ce5c04e0f9e1e457a7b65a891)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:42.906103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:42.923867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/292] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:55:42 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/users/04246b6923fc46c79c520fac5b3ceea7/roles/0777af4ce5c04e0f9e1e457a7b65a891 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:43.045789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20a33c-883b-4163-9434-bd1fbe96c298 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:43.094245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:43.095097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.095392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.095671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.096947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:43.098573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39230b-2f31-452a-a22f-834adb670770 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:43.100367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/293] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:42 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:43.106846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:43.123706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:43.240739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ade9a3-1694-46e6-a984-51cc49bfa9cd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:43.277264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:43.278210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.278691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.279100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.280075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=adb6fc4cad6e4372a5abbf970610f950) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:43.280587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:43.282172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4372f1-ee19-4ae9-8558-cd58fec88cfe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:43.288996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/294] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:43 2022] GET /identity/v3/role_assignments?user.id=adb6fc4cad6e4372a5abbf970610f950&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=de0ee47deb904e17990632bc8067f864 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:43.853283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6996f-653b-485c-afed-cdc36b754776 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.853679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6996f-653b-485c-afed-cdc36b754776 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.853858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6996f-653b-485c-afed-cdc36b754776 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.855480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/295] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:43.860979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.861270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.861547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.881507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:43.882719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:43.934746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5a13-ec13-435c-992a-41b0b8b09e25 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.935061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5a13-ec13-435c-992a-41b0b8b09e25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.935306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5a13-ec13-435c-992a-41b0b8b09e25 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.941217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/296] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:43.947994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:43.948693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:43.949241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:43.959726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:43.967496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/297] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:43.974967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:43.976461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:43.984198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:43.998300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.065829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/298] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:44.089309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.094933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1743e-2126-49ae-851f-8b332160a166 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.132254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.133046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.133279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.133554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.144875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:44.154130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:44.155032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:44.155820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/299] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:44.161659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.197256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0aed8-5658-4830-9dd8-a0adf7a1b1e9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.228624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.235212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.236432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.237021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.237549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.239184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.241000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/300] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:44.253129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.276455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a128a-5d68-4e25-b554-b511ec2c3bdf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.277218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.316707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.317392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.317678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.317955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.318820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:44.319232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.320659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf859d-b6e4-4b3b-8361-e73bfad2f0b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.331472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/301] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:44.333728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248895b-17f6-4043-b5ed-88d30c8fb434 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.334015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248895b-17f6-4043-b5ed-88d30c8fb434 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.334291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248895b-17f6-4043-b5ed-88d30c8fb434 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.335922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/302] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:44.337382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248895b-17f6-4043-b5ed-88d30c8fb434 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.360652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080e0b-c306-4c96-9bf6-935395a89d57 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.395046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.395685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.395984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.396318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.397040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.398537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a0d82-1af2-4b62-b127-f6b3a364a9e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.414008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/303] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Jun 21 02:55:44 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:44.415239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.415525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.415799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.437916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:44.439230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:44.466604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248895b-17f6-4043-b5ed-88d30c8fb434 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.499808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.515482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/304] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:44.518381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.519611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.520092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.520581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] PATH_INFO: `/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.537258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] RBAC: Authorizing `identity:get_user(user_id=90a4b44246d94ee5b69ad6aa8eeb5e2b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.538339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.539338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.542567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/305] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:44.548938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.551927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.671772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89765f7d-fba5-4b57-9b1a-603cf65fa765 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.676076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3d2ea-49c3-432b-929a-ec1f1e0f1255 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.715429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.716242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.716556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.716831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.717637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.718498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.718963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.719423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.720564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, user.id=90a4b44246d94ee5b69ad6aa8eeb5e2b, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:44.721067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.722827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.728352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:44.729413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/306] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/role_assignments?user.id=90a4b44246d94ee5b69ad6aa8eeb5e2b&scope.system=all&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:44.736505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:44.737453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:44.738373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/307] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:44.744271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.798499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:44.859562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f077d1-d523-47d2-b676-568e71272f41 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:44.898561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:44.899506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:44.899918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:44.900335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:44.901295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:44.901741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:44.903315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:44.913244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/308] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:44.918646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:44.993536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:45.028433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee2c7f-97cc-48c6-a5f6-26c55b953b05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:45.063473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:45.064717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.065303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.065788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] PATH_INFO: `/v3/users/860b1ac67ec9407d8a75a1f14edda391` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.067204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] RBAC: Authorizing `identity:get_user(user_id=860b1ac67ec9407d8a75a1f14edda391)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:45.069318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71464-8c59-4625-80ce-58e9f20b2d58 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:45.071499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/309] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:44 2022] GET /identity/v3/users/860b1ac67ec9407d8a75a1f14edda391 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:45.077136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:45.182995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a26ae-9bb9-47c5-a572-a37d37d9f268 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:45.192796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:45.218314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:45.219243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.219526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.219806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] PATH_INFO: `/v3/system/users/860b1ac67ec9407d8a75a1f14edda391/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.221613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=860b1ac67ec9407d8a75a1f14edda391, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:45.223365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17020-d3be-4d6e-8741-59e22a197474 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:45.234492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/310] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Jun 21 02:55:45 2022] PUT /identity/v3/system/users/860b1ac67ec9407d8a75a1f14edda391/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:45.250606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205e80e-8619-4279-b400-192a0922c9be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.251372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205e80e-8619-4279-b400-192a0922c9be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.251848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205e80e-8619-4279-b400-192a0922c9be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.254446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/311] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:45.260628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.260940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.261191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.279805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:45.281483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:45.362392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/312] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:45.387311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205e80e-8619-4279-b400-192a0922c9be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:45.499627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205e80e-8619-4279-b400-192a0922c9be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:45.535755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:45.536900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.537342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.537756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.545381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:45.553311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:45.561392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:45.562303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:45.563076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:45.565548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/313] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:45.571548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:45.674388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:45.678068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864242d-a97c-4f93-913f-2cc3ec2cf0db None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:45.714057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:45.714735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.715015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.715290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.716040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:45.716347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:45.718033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:45.727056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/314] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:45.732623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:45.836882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201f68-d5e7-4bbc-a33e-190602c39dcb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:45.871379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:45.872634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:45.873116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:45.873608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] PATH_INFO: `/v3/users/04246b6923fc46c79c520fac5b3ceea7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:45.875344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] RBAC: Authorizing `identity:get_user(user_id=04246b6923fc46c79c520fac5b3ceea7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:45.877059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:45.879428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/315] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:45 2022] GET /identity/v3/users/04246b6923fc46c79c520fac5b3ceea7 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:45.885194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:45.994425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9bfbd-64f8-44ae-9c34-6f750bf34309 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:46.031137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:46.031795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.032076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.032351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.065542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:46.104158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:46.105585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:46.203650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/316] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:46.209849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:46.277486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:46.313182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546fe85-2886-415a-b310-5d8151eff6de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:46.347426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:46.348392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.348706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.348980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.350201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:46.351676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbae46-fa9c-4706-aaf7-172fac055776 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:46.353756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/317] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:46 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:46.359802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:46.422584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:46.465345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255a15f-ecf8-4d06-ace8-fa5093a24253 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:46.500649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:46.501327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.501610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.501884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.502725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, scope.project.id=8c27da0908c54bf580f79eb1d2dab577, user.id=04246b6923fc46c79c520fac5b3ceea7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:46.503036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:46.504852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49543e93-f442-498a-84b0-626e204ba228 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:46.511375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/318] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:55:46 2022] GET /identity/v3/role_assignments?user.id=04246b6923fc46c79c520fac5b3ceea7&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:46.528833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aed8f-3ef7-45d2-a607-003a114e1e25 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.529148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aed8f-3ef7-45d2-a607-003a114e1e25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.529397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aed8f-3ef7-45d2-a607-003a114e1e25 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.530904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/319] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:46.536952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.537241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.537523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.556024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:46.557471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:46.632297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/320] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:46.655418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aed8f-3ef7-45d2-a607-003a114e1e25 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:46.762080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462aed8f-3ef7-45d2-a607-003a114e1e25 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:46.796896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:46.798099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.798577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.799066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.800797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:46.802481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22802b85-503f-4a4b-9bc4-1bb7929b3c1b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:46.804341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/321] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:46.810121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:46.869246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:46.915038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d6512-88c8-4933-918c-21a265c62ce3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:46.947257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7abe-15ef-467f-87c0-54ed9dbba6b8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.947524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7abe-15ef-467f-87c0-54ed9dbba6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.947792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7abe-15ef-467f-87c0-54ed9dbba6b8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.949399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/322] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:46.952448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:46.952448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.952448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.952448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.952750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:46.953656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:46.954609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:46.954879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:46.955155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:46.960616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:46.971413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/323] 10.222.0.16 () {60 vars in 1134 bytes} [Tue Jun 21 02:55:46 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:55:46.974809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:46.976089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:47.055121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/324] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:47.077931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.193596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c41e-0fee-480d-b2a2-2b69a78432e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.212854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:47.234474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.235311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.235601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.235885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.247954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ee4d2b00-4ffd-46b3-ac86-a4b112d6cb08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:47.257071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:47.257982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:47.259029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/325] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:47.266108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.386372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e11b4-a32c-4fe4-a6f1-68d8904138ef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.386661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e11b4-a32c-4fe4-a6f1-68d8904138ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.386945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e11b4-a32c-4fe4-a6f1-68d8904138ef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.388086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33e287-9a4f-4e64-b415-2fe9cba6f269 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.393030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/326] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:47.400076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d767b1-f116-4e63-815a-02a39040149c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.400511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d767b1-f116-4e63-815a-02a39040149c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.400798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d767b1-f116-4e63-815a-02a39040149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.423221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-37d767b1-f116-4e63-815a-02a39040149c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:47.425031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37d767b1-f116-4e63-815a-02a39040149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:47.427046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:47.435985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.437378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.437680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.437974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.438770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:47.439092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:47.441667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:47.454379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/327] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:47.460031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.523247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/328] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:47.549596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d767b1-f116-4e63-815a-02a39040149c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.578456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963a869-9033-4473-9a27-4fc7e8180702 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.615518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.616448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.616740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.617012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] PATH_INFO: `/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.618238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] RBAC: Authorizing `identity:get_user(user_id=90a4b44246d94ee5b69ad6aa8eeb5e2b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:47.619715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:47.621723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/329] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:47.628047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.669279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d767b1-f116-4e63-815a-02a39040149c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.679016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:47.702338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:47.710831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.712861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.713360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.713842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.726876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:47.735687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:47.736621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:47.737423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:47.740780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/330] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:47.746660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:47.747280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e70b65-7f3e-4817-ad4b-19daa5aed389 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.788835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.789732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.790022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.790301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] PATH_INFO: `/v3/system/users/90a4b44246d94ee5b69ad6aa8eeb5e2b/roles/0777af4ce5c04e0f9e1e457a7b65a891` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.791843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=90a4b44246d94ee5b69ad6aa8eeb5e2b, role_id=0777af4ce5c04e0f9e1e457a7b65a891)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:47.793297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:47.814286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/331] 10.222.0.16 () {58 vars in 1334 bytes} [Tue Jun 21 02:55:47 2022] PUT /identity/v3/system/users/90a4b44246d94ee5b69ad6aa8eeb5e2b/roles/0777af4ce5c04e0f9e1e457a7b65a891 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:47.867125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8d005-56df-4153-967e-bb44b68d825e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:47.913185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:47.914272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:47.914745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:47.915229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:47.916336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:47.916875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:47.918501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:47.929631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/332] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:47.935678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:48.092823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8e37a-8b20-4a14-b150-9f697836b8db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:48.121698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:48.132630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:48.133526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:48.133818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:48.134134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] PATH_INFO: `/v3/users/860b1ac67ec9407d8a75a1f14edda391` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:48.135966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] RBAC: Authorizing `identity:get_user(user_id=860b1ac67ec9407d8a75a1f14edda391)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:48.137850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d585-0381-4ca6-a93a-9da4b8db3d28 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:48.139830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/333] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:47 2022] GET /identity/v3/users/860b1ac67ec9407d8a75a1f14edda391 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:48.145842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:48.177629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:48.267161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9911-9f65-497f-ab00-92ec6086ea2b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:48.311668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:48.312340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:48.312646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:48.312921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:48.313758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] RBAC: Adding query filter params (role.id=de0ee47deb904e17990632bc8067f864, user.id=860b1ac67ec9407d8a75a1f14edda391, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:48.314068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:48.315514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba82ed-6705-4beb-ace7-e1b92d086edc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:48.331994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/334] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jun 21 02:55:48 2022] GET /identity/v3/role_assignments?user.id=860b1ac67ec9407d8a75a1f14edda391&scope.system=all&role.id=de0ee47deb904e17990632bc8067f864 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:49.902566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125023f9-bead-47c2-be19-ab4bd29d8ce9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:49.903166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125023f9-bead-47c2-be19-ab4bd29d8ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:49.903166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125023f9-bead-47c2-be19-ab4bd29d8ce9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:49.904809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/335] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:49.909995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535609-a986-4149-9a8b-ef0ae6e4e2ee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:49.910451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535609-a986-4149-9a8b-ef0ae6e4e2ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:49.910547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535609-a986-4149-9a8b-ef0ae6e4e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:49.934245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4535609-a986-4149-9a8b-ef0ae6e4e2ee None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:49.935475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4535609-a986-4149-9a8b-ef0ae6e4e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:50.008240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/336] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:50.044339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125023f9-bead-47c2-be19-ab4bd29d8ce9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:50.187646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125023f9-bead-47c2-be19-ab4bd29d8ce9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:50.212351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dbf0-37a6-4d25-af3a-2dd29f7f2df6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.212726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dbf0-37a6-4d25-af3a-2dd29f7f2df6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.212947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dbf0-37a6-4d25-af3a-2dd29f7f2df6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.214612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/337] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:50.219486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555812-1e5e-4b71-92af-093d1cf45557 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.219776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555812-1e5e-4b71-92af-093d1cf45557 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.220056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07555812-1e5e-4b71-92af-093d1cf45557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.234010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:50.234834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.235119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.235397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.259889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:50.263692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-07555812-1e5e-4b71-92af-093d1cf45557 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:50.264953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07555812-1e5e-4b71-92af-093d1cf45557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c45eda1e-0878-45a4-a51d-0393870698c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:50.267844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:50.268802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:50.269979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:50.270633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/338] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:50.271408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0317cc-6fa4-4e1c-877b-280da3b28149 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.271698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0317cc-6fa4-4e1c-877b-280da3b28149 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.271983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0317cc-6fa4-4e1c-877b-280da3b28149 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.278429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/339] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:50.280019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0317cc-6fa4-4e1c-877b-280da3b28149 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:50.289237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:50.382393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:50.400651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/340] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:50.401979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.402266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.402543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.451569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:50.452992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:50.484591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0317cc-6fa4-4e1c-877b-280da3b28149 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:50.555514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:50.556203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.556504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.556779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.558378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:50.558890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:50.560311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:50.585067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/341] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:50.585438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/342] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:50.586148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:50.601962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:50.716266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:50.738049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c34d-667f-4ccc-ad40-2d283427482d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:50.740147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28fcf8-2ee1-43c2-812b-1158aa3a4725 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:50.756693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:50.783212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:50.783884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.784167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.784993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.787002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:50.787810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.788092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.788367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.811466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:50.825054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:50.828449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/343] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:50.830421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:50.938094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a5c91-7aae-429e-9404-d86841b68256 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:50.968216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:50.969683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:50.978268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:50.979102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:50.979388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:50.979667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:50.996838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.007322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:51.015452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:51.017623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:51.019650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:51.024499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/344] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:51.027716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.141394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929810e-c405-4e0d-9add-01741e337618 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.168454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/345] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:55:51.168454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.178595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.179283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.179567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.179854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.180791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:51.181101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.182522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.193685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/346] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:51.194678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.197306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.288923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.310133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf89eed-9f45-4fc8-9d49-fc036a2150a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.344359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cb1a7-82a8-493b-98d0-492d59df964d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.356280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.356998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.357279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.357554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.358314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:51.358620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.360012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7f228-e3c2-4f8b-a817-3bb52207ac43 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.369806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/347] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:51.371082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1645162-f163-494e-ad1f-7142bff8101b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.371309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1645162-f163-494e-ad1f-7142bff8101b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.371584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1645162-f163-494e-ad1f-7142bff8101b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.372859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/348] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:51.373633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1645162-f163-494e-ad1f-7142bff8101b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.387225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.388461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.388748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.389059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.390219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.391827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa2f2d-7ee6-4194-ade0-b9e2dacbe46b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.393523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/349] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:51.394837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbba6c-8ce5-4209-9e51-d4ed0f453e5b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.395124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbba6c-8ce5-4209-9e51-d4ed0f453e5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.395405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbba6c-8ce5-4209-9e51-d4ed0f453e5b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.396956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/350] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:51.398093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.398478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.398887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.419939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:51.421183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:51.489647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.508798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/351] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:51.509943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.528761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1645162-f163-494e-ad1f-7142bff8101b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.581147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.582761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.583661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.583661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.583851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.638613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb653e38-3aeb-4778-9deb-32424896cccb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.668646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.670136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.683866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.684588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.684874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.685154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.776046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.790189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.793568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394b102-aabc-4f7b-a1dd-8a89680639df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.811394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/352] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:51.812284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:51.858897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:51.917174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/353] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:51.918879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.920080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.920080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.934115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca283-6e88-4b94-8d6e-4e5c1169ad9f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:51.939250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:51.940511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:51.971873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:51.972909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:51.973193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:51.973467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:51.981730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:51.983392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:51.985317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/354] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:51.986300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.017167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/355] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:52.018044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.040829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.109523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166137c-d858-4327-947c-ee9bd1ddf100 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.117175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.138965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadd81b-f009-4232-a246-6491f9412708 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.150681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.151153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.151441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.151719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.167729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9cc550f8-07d9-4bca-aca4-63e26fb3fa3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:52.176081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:52.177054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 02:55:52.177902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:52.178854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/356] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:52.179524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4c2a3-8060-4ee8-8b6e-080203fdecd6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.179905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4c2a3-8060-4ee8-8b6e-080203fdecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.180185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4c2a3-8060-4ee8-8b6e-080203fdecd6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.183143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/357] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:52.183143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4c2a3-8060-4ee8-8b6e-080203fdecd6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.185032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.185899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.186174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.186443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.187811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.189276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.191720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/358] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.192363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.304973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd91efc-6882-4001-a4df-bf3680b182fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.306126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4c2a3-8060-4ee8-8b6e-080203fdecd6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.309608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.346481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.347673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.347957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.348238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.349127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=efdfe2f79f234724a0bdac8105ff3707) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:52.349446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.350900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.351767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.352988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.353268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.353539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.355501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.356947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/359] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=de0ee47deb904e17990632bc8067f864 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.357066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.357605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.358826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/360] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.363479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.388840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.486756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be4bcb-02fc-4ac8-8813-0a9ab97b3df0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.496442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573862e-78db-422f-885d-bdc14b021cad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.544245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.544828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.545517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.545626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.545788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.545885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.546062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.546159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.546824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:52.547123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.548564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e06a-1b9f-4940-a445-4e7844f62904 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.558890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/361] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.560777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.561553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.562039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.562149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:52.571122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:52.572062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:52.572904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:52.574738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/362] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:52.575325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.584712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:52.585915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:52.614941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.667629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/363] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:52.668300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.678935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.703332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e792a69-a8ff-414c-b5f6-a5a2635851f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.740063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.741326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.741826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.742301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.743679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.745379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.747033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/364] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.747878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.789574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9179a0-1ad7-46aa-8120-05e8321016ab None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.834518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.835716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.836163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.836654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.838030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.839804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.841627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/365] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.842406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.867858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7db34-2a0e-42ce-bcd3-3148f63d5bb7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:52.886492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.916513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:52.917411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:52.917740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:52.917984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] PATH_INFO: `/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:52.919197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] RBAC: Authorizing `identity:get_user(user_id=90a4b44246d94ee5b69ad6aa8eeb5e2b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:52.920731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:52.923155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/366] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/users/90a4b44246d94ee5b69ad6aa8eeb5e2b => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:52.923475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:52.944582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:52.969765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63141760-cc58-40e0-912c-d3ea725e6093 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.018415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.019487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.019723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.020037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.020922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:53.021331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.022772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.032828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/367] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:53.033682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.058664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1855-edc5-4a64-8e52-bcf01433c78c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.108870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.109693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.109980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.110249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.124330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:53.133156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:53.134045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:53.134844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/368] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:53.135506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.158631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.172150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a3535-b530-4bf7-99a6-bac13a2f029f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.206183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.207116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.207401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.207671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.208544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=8c27da0908c54bf580f79eb1d2dab577) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:53.208856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.210326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.215979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/369] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=de0ee47deb904e17990632bc8067f864 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:53.216568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.233432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.268183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88737dbe-3e6c-4f14-8306-6cfbb9b28644 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.323629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.324302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.324622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.324897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.325744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=692ec53a198846cda9892422cde67a24, scope.project.id=efdfe2f79f234724a0bdac8105ff3707) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:53.326055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.327522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.340881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/370] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=692ec53a198846cda9892422cde67a24 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:53.341999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.353620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4da545-0d5b-4ca9-b8b8-e7c2c288b509 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.398979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.399989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.400492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.401449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.402530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] RBAC: Adding query filter params (role.id=0777af4ce5c04e0f9e1e457a7b65a891, user.id=90a4b44246d94ee5b69ad6aa8eeb5e2b, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:53.403069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.404881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.420999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/371] 10.222.0.16 () {56 vars in 1308 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/role_assignments?user.id=90a4b44246d94ee5b69ad6aa8eeb5e2b&scope.system=all&role.id=0777af4ce5c04e0f9e1e457a7b65a891 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:53.422316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.442330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.551802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e9c8e-6f3e-4b44-ab60-e4113e6a676a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.595257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.596338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.596892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.597404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.598534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:53.599102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.600794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.617017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/372] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:53.619325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:53.643162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a64f-77f2-4c1e-a2a4-24fca25be91e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.716790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.717877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.718543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.718877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.719159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.733762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.743095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93919626-c194-4b5c-b766-9439efbf96e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:53.783123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:53.784806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:53.785315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:53.785816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:53.866084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.867927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.919395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:53.921366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:53.957091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:53.988633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/373] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:55:53.990363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.100021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0df2f-20be-49e9-9c25-50b63f8c1fb7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.154332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.155211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.155494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.155768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] PATH_INFO: `/v3/projects/eab06a8b789343a7bc8eebc83d2a6637` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.157751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] RBAC: Authorizing `identity:get_project(project_id=eab06a8b789343a7bc8eebc83d2a6637)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.159230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.163103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/projects/eab06a8b789343a7bc8eebc83d2a6637 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:54.168710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.195404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/375] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 854 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:54.201619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.251950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:54.271145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:54.355073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95418-da2d-47a6-ae4d-23936db891c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.383772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d62e77-2391-4719-9be0-5e3fb4828878 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.418335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.419231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.419514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.419791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] PATH_INFO: `/v3/groups/4765a54924c44a17be638852a8886936` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.431712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] RBAC: Authorizing `identity:get_group(group_id=4765a54924c44a17be638852a8886936)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.433226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.435416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/376] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/groups/4765a54924c44a17be638852a8886936 => generated 229 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:54.449234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.458449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.461005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.461288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.461567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.463347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.464846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.470843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/377] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:54.474702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.585021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139c0ed-d4be-401d-b9b8-685d3144dc64 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.597187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:54.631548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.632616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.633121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.633322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d604c5-7871-4ea1-a6b8-29b071a5f774 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.633789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.634985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] RBAC: Adding query filter params (group.id=4765a54924c44a17be638852a8886936, role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=eab06a8b789343a7bc8eebc83d2a6637) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:54.635489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.637982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de734ff7-d5fb-4ccf-a2ea-780e384d1cad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.641562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:54.646131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/378] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/role_assignments?group.id=4765a54924c44a17be638852a8886936&scope.project.id=eab06a8b789343a7bc8eebc83d2a6637&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:54.647993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62458-87dc-4de5-8881-2c1759c70111 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.648534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62458-87dc-4de5-8881-2c1759c70111 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.649171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62458-87dc-4de5-8881-2c1759c70111 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.651806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/379] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:54.661037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.661648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.662131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.680617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.681553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.681882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.682208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.683449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.684901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:54.685134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.686539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:54.686894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/380] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:54.692882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.821112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/381] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:54.858563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b619-221c-4322-8d3b-a12755d95a9b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:54.888851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:54.921624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:54.922717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:54.923197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:54.923689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:54.924853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=692ec53a198846cda9892422cde67a24, scope.project.id=8c27da0908c54bf580f79eb1d2dab577) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:54.926102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:54.927782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:54.934347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/382] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=692ec53a198846cda9892422cde67a24 => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:55.038913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335ed83-23c1-44c1-a827-b611ae94c820 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:55.046528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:55.101327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:55.102986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.103491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.103985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.122785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:55.132382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:55.133358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:55.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:55.142563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/383] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:55:55.148345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:55.215128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:55.311326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76313106-bdee-4181-8d45-9770e82f692a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:55.355860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:55.356569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.356855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.357136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.357896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:55.358207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:55.359594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:55.372750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/384] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:55.377864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:55.541605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94648b-bf25-4d0f-b3cd-291f0ebc4967 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:55.587284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:55.588226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.588550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.588876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.590105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:55.591645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6850ea-0f25-4a70-9d95-7169a2555d6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:55.593654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/385] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:55 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:55.598932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:55.663568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9761c5-2ca1-484b-b72c-aae61f565390 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.663862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9761c5-2ca1-484b-b72c-aae61f565390 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.664041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9761c5-2ca1-484b-b72c-aae61f565390 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.665536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/386] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:55.672717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.673229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.673706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.704952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:55.706767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:55.752628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:55.760028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b47f-bf40-4032-b731-4c11f7795078 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:55.768778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:55.822712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:55.823963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:55.824453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:55.824937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:55.827490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:55.829199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:55.834400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/388] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:55.834678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/388] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:55 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:55.837129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:55.873422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:55.991223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116b3b1-63a7-4e71-bafa-d3a78c98f4a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.012621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991fb1-fe35-494e-8c31-76854e694d0d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.029447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.030440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.030727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.031004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/groups/e7b57f6904294dc98b674961d14553c5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.033009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, group_id=e7b57f6904294dc98b674961d14553c5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.034611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280b64c-2e4e-4c75-bbfe-ce7f543572d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.049172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.050375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.050856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.051261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/389] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jun 21 02:55:55 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/groups/e7b57f6904294dc98b674961d14553c5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:56.051611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.052829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d90286-3bf5-44df-a706-a641eefb2ccc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.053125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d90286-3bf5-44df-a706-a641eefb2ccc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.053501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d90286-3bf5-44df-a706-a641eefb2ccc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.059424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/390] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:56.061306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.061601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.061887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.066793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:56.075508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:56.076522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:56.077412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:56.080241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/391] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:56.080558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:56.082494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:56.090616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.113581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.172927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/392] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:56.192168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.193712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.287903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe1c00-554b-4cfc-b20c-e84ee41ea507 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.312396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b33fc-5590-4143-bdb1-e0df2a002870 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.324511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.325237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.325614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.325982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.326924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:56.327326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.328821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.338376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/393] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:56.344405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.358811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.360027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.361202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.361684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.375685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:56.386181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:56.387194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:56.388003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:56.390762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/394] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:56.397752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.414412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.461948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.475589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a7455-ae84-4e42-bca6-18e35b015f70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.526397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1d11-0a52-4c91-a227-4bc3bb21f2d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.567495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.569419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.570518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.571279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.572675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:56.573137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.574684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.583997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/395] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:56.588774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.588774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.588774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.588774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.589128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.589128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.590180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.590921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/396] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:56.596634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.744326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4029fe7-82d0-4fc2-8b17-ae08581a82ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.750459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.803543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58155c3c-67fa-42f0-a0cd-b90548a4e8cc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:56.806080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.806959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.807244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.807520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.808843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.810297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.812179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/397] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:56.819053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.865732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:56.870009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:56.871005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:56.871289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:56.871574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:56.873836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:56.875630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:56.877468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/398] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:56.883696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:56.992126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f585be-3f4b-4131-bab1-cac0aaf04b14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:57.028365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:57.029673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.030162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.030652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/groups/e7b57f6904294dc98b674961d14553c5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.033051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, group_id=e7b57f6904294dc98b674961d14553c5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:57.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:57.052491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/399] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jun 21 02:55:56 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/groups/e7b57f6904294dc98b674961d14553c5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:57.092672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710c52-68f5-4ff9-b310-da9b11631a59 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:57.156639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:57.157537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.157821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.158097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.165411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:57.166947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8ca7e-3ed2-49e5-9fae-7279455d71ed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:57.169184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/400] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:56 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:57.174951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:57.244507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a8d96-f9cc-4c7a-adbd-718b78482e0d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.244825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a8d96-f9cc-4c7a-adbd-718b78482e0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.245081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a8d96-f9cc-4c7a-adbd-718b78482e0d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.247038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/401] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:57.256200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.256563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.256793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.289028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:57.290240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:57.292383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:57.320778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:57.336439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e6fa3-a314-4063-a2a1-860fa2874c26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:57.378369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/402] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:57.401099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:57.402242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.402688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.403108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707/groups/e7b57f6904294dc98b674961d14553c5/roles/692ec53a198846cda9892422cde67a24` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.409795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] RBAC: Authorizing `identity:create_grant(project_id=efdfe2f79f234724a0bdac8105ff3707, group_id=e7b57f6904294dc98b674961d14553c5, role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:57.411414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b89aa-5439-4170-b094-9f8474a5f91f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:57.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:57.445656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/403] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jun 21 02:55:57 2022] PUT /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707/groups/e7b57f6904294dc98b674961d14553c5/roles/692ec53a198846cda9892422cde67a24 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:57.547175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f580fb7-3439-4def-b557-34eb18bee70b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:57.560886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a397f7-c09d-4abb-934c-8a4582c7f1a7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.561226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a397f7-c09d-4abb-934c-8a4582c7f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.561450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a397f7-c09d-4abb-934c-8a4582c7f1a7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.563093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/404] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:55:57.568510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.568806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.569087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.601722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:57.602962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:57.618875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:57.619701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.619984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.620259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.634219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:57.641940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:57.654671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:57.655704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:57.656524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/405] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:55:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:57.663971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:57.752897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/406] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:55:57.777518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:57.798384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b624-a986-4212-9ca1-5f63635ef65e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:57.869627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:57.870312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:57.870598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:57.870876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:57.871643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:57.871953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:57.873376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:57.889814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/407] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:57.901529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:57.910251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:57.938601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a3b3-8ea3-46dc-b73c-a6b1a1774406 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.002396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.004521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.005166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.006727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.023968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:58.037364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:58.045650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:58.046521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:55:58.047303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/408] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:55:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:58.053489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.076249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21561b-99ca-4c6b-9287-582da9d7d610 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.141794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.143632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.144289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.151323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] PATH_INFO: `/v3/groups/4765a54924c44a17be638852a8886936` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.153946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] RBAC: Authorizing `identity:get_group(group_id=4765a54924c44a17be638852a8886936)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.158165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.166209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/409] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:57 2022] GET /identity/v3/groups/4765a54924c44a17be638852a8886936 => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:58.172668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.268613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d2bd-3e40-45e1-8d3d-281bf344e0ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.292094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0767a-f136-4209-9549-95ea97827bc6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.319927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:58.331314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357096-348b-434f-9460-357f25528de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.332131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357096-348b-434f-9460-357f25528de6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.332722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357096-348b-434f-9460-357f25528de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.333144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357096-348b-434f-9460-357f25528de6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.336431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357096-348b-434f-9460-357f25528de6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:58.336431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357096-348b-434f-9460-357f25528de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.336431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357096-348b-434f-9460-357f25528de6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.338547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:58.341987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.342872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.343154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.343428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] PATH_INFO: `/v3/projects/eab06a8b789343a7bc8eebc83d2a6637` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.346733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/410] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:58.349588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=eab06a8b789343a7bc8eebc83d2a6637)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.351604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.354518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357096-348b-434f-9460-357f25528de6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.358932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/411] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/projects/eab06a8b789343a7bc8eebc83d2a6637 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:58.364148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.539807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357096-348b-434f-9460-357f25528de6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.542345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996039a3-13aa-4ab2-ab79-cde55066b5a7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.580499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.582111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.582400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.582678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] PATH_INFO: `/v3/projects/eab06a8b789343a7bc8eebc83d2a6637/groups/4765a54924c44a17be638852a8886936/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.584930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] RBAC: Authorizing `identity:create_grant(project_id=eab06a8b789343a7bc8eebc83d2a6637, group_id=4765a54924c44a17be638852a8886936, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.585111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.585971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.586255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.586474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d1b8b-58d0-42a7-9031-86394463c811 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.586571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.587717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.589195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a397d90-ea22-4d40-a627-eb39eac4ec16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.592511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/412] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:58.594017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d18f0-8da5-4c4d-80d4-f4fb452f1aff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.594310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d18f0-8da5-4c4d-80d4-f4fb452f1aff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.594590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d18f0-8da5-4c4d-80d4-f4fb452f1aff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.596484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/413] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:58.597401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d18f0-8da5-4c4d-80d4-f4fb452f1aff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.607993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/414] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jun 21 02:55:58 2022] PUT /identity/v3/projects/eab06a8b789343a7bc8eebc83d2a6637/groups/4765a54924c44a17be638852a8886936/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:55:58.609439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.609731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.610152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.628739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:58.630179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:58.677924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:58.689254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:58.710168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/415] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:58.733098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d18f0-8da5-4c4d-80d4-f4fb452f1aff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.737370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.820303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.821223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.821574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.821853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.823234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:58.825318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:58.826971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/416] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:58.832231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.889488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721bbf7-9f05-4a5c-8479-5742c72f9b94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:58.947018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:58.948806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:58.949093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:58.949365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:58.970650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:55:58.979112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:55:58.980030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:55:58.980845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/417] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:55:58.986169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:58.991599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b8dfa-3ea6-44a6-a895-7cffc9f0196a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:59.043930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:59.046250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.047676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.049280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577/groups/e7b57f6904294dc98b674961d14553c5/roles/692ec53a198846cda9892422cde67a24` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.053715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c27da0908c54bf580f79eb1d2dab577, group_id=e7b57f6904294dc98b674961d14553c5, role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:59.057690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:59.060619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:59.093548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/418] 10.222.0.16 () {58 vars in 1442 bytes} [Tue Jun 21 02:55:58 2022] PUT /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577/groups/e7b57f6904294dc98b674961d14553c5/roles/692ec53a198846cda9892422cde67a24 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 02:55:59.094384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:59.158120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a22d0-2b46-4b9b-af7b-71b0d668a25c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:59.209320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:59.210932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.211414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.211880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.212948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:55:59.213467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:59.215226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:59.232186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/419] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:55:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:55:59.238236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:59.373002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8de92-3f80-4c57-9afa-d4290e906d40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:59.412792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:59.413456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.413743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.414025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.477263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:59.485120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:59.486631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2eb03-ee5b-4fb2-bfbe-4f3555c082af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:59.590339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/420] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:55:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:55:59.595544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:59.656041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9fa1-5a22-4777-bd56-87d05a8a2ca3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.656684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9fa1-5a22-4777-bd56-87d05a8a2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.657028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a9fa1-5a22-4777-bd56-87d05a8a2ca3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.658722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/421] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:55:59.664567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.664855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.665131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.697508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:55:59.698726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:55:59.727674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc528687-6d17-4b31-97d5-6902d65aec25 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:59.740156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:59.780305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:55:59.781230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:55:59.781490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:55:59.781758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:55:59.782964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:55:59.784603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:55:59.786484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/422] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:55:59 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:55:59.800525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:59.802187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/423] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:55:59.841527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:55:59.884807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:55:59.968839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9f065-4a72-4c57-85f9-0fb596f8a0da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:55:59.969464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2235-9488-457d-8906-b3646b610db6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.006009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.006854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.007133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.007406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.014356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.015721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.016555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.017047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.018744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.019235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:00.020732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37bcc4-177e-4b6e-9a53-3e3df7c628ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:00.027388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:00.028305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 02:56:00.029126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:00.030244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/425] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:55:59 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:00.031822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59dc-82b0-471a-9cf5-1cb609d55273 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.032302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59dc-82b0-471a-9cf5-1cb609d55273 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.032788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d59dc-82b0-471a-9cf5-1cb609d55273 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.033307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/425] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:55:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:00.034127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.034984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/426] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:00.035888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d59dc-82b0-471a-9cf5-1cb609d55273 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.093994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.139725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.157486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7463ad-e68d-477d-a3ca-a2c1da3f30b3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.165956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d59dc-82b0-471a-9cf5-1cb609d55273 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.194617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.195441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.195918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.196311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.197364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=efdfe2f79f234724a0bdac8105ff3707) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:00.197778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.199466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde93f1-afb0-47d7-a828-c4ef38ec667c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.205601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.206673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.206960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.207286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.208049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:00.208422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.209916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.211675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/427] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=de0ee47deb904e17990632bc8067f864 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:00.213279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.213571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.213854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.220068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/428] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:00.225949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.253195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:00.254629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:00.381285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb584-3d5c-4378-8823-c1d0ede5a1fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.398460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.427172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.427953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.428243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.428579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.431314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/429] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:00.460306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.535941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.538339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.572354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.608207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441807f3-3ace-47f7-8aec-1b7baade0055 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.650491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/430] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:56:00.653243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.654074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.654354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.654750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.655634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.666615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.671052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:00.680703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:00.682919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:00.684828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/431] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:00.694541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.765380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796792e-2e0f-4cf3-b064-8b1cfa45ad9c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.801176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.801978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.802263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.802364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6669e7-c355-4523-a3b8-fb225144cf03 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.802536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.804034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.805835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.808012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/432] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:00.813338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.852017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.853288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.853793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.854517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.855709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:00.856191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.858814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.865376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.873601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/433] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:00.882523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:00.907179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:00.938312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e125a-2bb3-4239-b8d4-09b964866cb9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:00.982510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:00.983633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:00.984019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:00.984378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:00.988611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:00.990222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:00.992109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/434] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:00.997637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.003368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cfd0a-cf8e-4fe3-97be-f6f6e71b0bc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.042686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.043701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.044178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.044707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.115554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca36d-74a1-4407-b824-17f5a8fdd6fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.128505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.131163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.132773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.162563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.163599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.164095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.164569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.165629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=de0ee47deb904e17990632bc8067f864, scope.project.id=8c27da0908c54bf580f79eb1d2dab577) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:01.165945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.167412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87ce9a-4243-48a3-8c5f-33d25c2f8825 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.173500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/435] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=de0ee47deb904e17990632bc8067f864 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:01.174769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae2d1-2cd7-435b-b4b1-b60d998c548e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.175391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae2d1-2cd7-435b-b4b1-b60d998c548e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.175673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae2d1-2cd7-435b-b4b1-b60d998c548e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.176997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/436] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:01.182761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.183050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.183321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.196046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.202470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:01.204348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:01.273073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/437] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:01.278149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.292894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/438] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:56:01.313974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.387466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.390858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b595d-8d46-44be-92f7-6fc53aa200d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.439706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73844fc-7422-4989-9ba0-90db5bab5e4c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.442060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.443053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.443414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.443694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] PATH_INFO: `/v3/projects/efdfe2f79f234724a0bdac8105ff3707` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.444941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] RBAC: Authorizing `identity:get_project(project_id=efdfe2f79f234724a0bdac8105ff3707)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.446413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.448129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/439] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/projects/efdfe2f79f234724a0bdac8105ff3707 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:01.453731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.477072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.477702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.477992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.478269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.481536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.490457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b5f11a86-b606-4942-aca1-3bd149e12afa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:01.498334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:01.499486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:01.500700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/440] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:01.502328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a9667-0759-424e-a0ce-4396082b25ea None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.503002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a9667-0759-424e-a0ce-4396082b25ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.503572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a9667-0759-424e-a0ce-4396082b25ea None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.505816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/441] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:01.506539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a9667-0759-424e-a0ce-4396082b25ea None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.558032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c572f8-a711-47ad-a83d-19baafa28724 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.591629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.592515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.592815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.593088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.594273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.595744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc767a5-aba5-441f-8d66-7796231bcffc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.597542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/442] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:01.598665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.598946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.599218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.609360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a9667-0759-424e-a0ce-4396082b25ea None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.615137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:01.616313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:01.644695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.645308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.645515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.645654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.645858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.646618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:01.646928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.648336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.657045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/443] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:01.657850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.687559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/444] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:01.688717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.717372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.759733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7ac95-4c0f-4d81-9a7a-940cb28b7977 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.788762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94f0a4-fa2d-4920-b0a1-9472441b9dca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.793775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.794441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.794724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.795000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.795814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=692ec53a198846cda9892422cde67a24, scope.project.id=efdfe2f79f234724a0bdac8105ff3707) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:01.796126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.797630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.804151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/445] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=efdfe2f79f234724a0bdac8105ff3707&role.id=692ec53a198846cda9892422cde67a24 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:01.804974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:01.841282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.841960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.842233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.842527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.884684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.918861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07bc36-1c48-4043-97e0-d14056b2af62 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:01.926672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:01.928323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:01.944962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:01.960012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:01.961225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:01.961731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:01.962008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:01.974843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:01.984513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:01.985503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:01.986304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/446] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:01.990795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.053170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/447] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:02.057808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.109181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05933679-996e-4089-b05c-a8ce39f260a2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.131559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.152617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.153546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.153865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.154149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.154927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:02.155239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.156716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.173077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/448] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:02.174618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.177254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe556-eead-4406-aded-4799f004e380 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.224040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.224947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.225291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.225564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] PATH_INFO: `/v3/projects/eab06a8b789343a7bc8eebc83d2a6637` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.226787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] RBAC: Authorizing `identity:get_project(project_id=eab06a8b789343a7bc8eebc83d2a6637)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.228255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.230269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.230389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/449] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/projects/eab06a8b789343a7bc8eebc83d2a6637 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:02.237613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.298279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230532e-9237-4bdb-94ac-19e34d6bda4e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.338922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.339886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.339886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.340117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.357759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d782-dd3f-4671-abbd-3effcbe9bb99 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.390807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.391664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.391943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.392217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] PATH_INFO: `/v3/groups/4765a54924c44a17be638852a8886936` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.393457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] RBAC: Authorizing `identity:get_group(group_id=4765a54924c44a17be638852a8886936)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.394978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.396649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/450] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/groups/4765a54924c44a17be638852a8886936 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:02.402340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.404951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.407823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.409360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.453512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.500820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3a9d-1c63-4855-9770-3a960ad12334 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.508457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/451] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:02.514109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.536219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.536883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.537166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.537449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.538305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] RBAC: Adding query filter params (group.id=4765a54924c44a17be638852a8886936, role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=eab06a8b789343a7bc8eebc83d2a6637) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:02.538602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.540039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.546164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/452] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/role_assignments?group.id=4765a54924c44a17be638852a8886936&scope.project.id=eab06a8b789343a7bc8eebc83d2a6637&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:02.614567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec6089-9e5f-4e89-9326-e61b7b3713df None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.652843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.653727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.654005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.654274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] PATH_INFO: `/v3/projects/8c27da0908c54bf580f79eb1d2dab577` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.655534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=8c27da0908c54bf580f79eb1d2dab577)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.657077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.658849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/453] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/projects/8c27da0908c54bf580f79eb1d2dab577 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:02.664725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.734947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.770258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8680db-6524-4720-a8f2-6455bcd2faaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.802442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.803297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.803580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.803861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] PATH_INFO: `/v3/groups/e7b57f6904294dc98b674961d14553c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.805039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] RBAC: Authorizing `identity:get_group(group_id=e7b57f6904294dc98b674961d14553c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.806494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f4eac-079b-4b70-863c-0062ac9bc6ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.808119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/454] 10.222.0.16 () {56 vars in 1180 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/groups/e7b57f6904294dc98b674961d14553c5 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:02.817784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:02.835398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:02.932286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ea857-e9ae-41cf-a054-3f3d7e89f2b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:02.967903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:02.968610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:02.968893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:02.969171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:02.970021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] RBAC: Adding query filter params (group.id=e7b57f6904294dc98b674961d14553c5, role.id=692ec53a198846cda9892422cde67a24, scope.project.id=8c27da0908c54bf580f79eb1d2dab577) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:02.970333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:02.971804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ab463-e68d-46ee-ba31-f7502bd8da11 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:02.979446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/455] 10.222.0.16 () {56 vars in 1376 bytes} [Tue Jun 21 02:56:02 2022] GET /identity/v3/role_assignments?group.id=e7b57f6904294dc98b674961d14553c5&scope.project.id=8c27da0908c54bf580f79eb1d2dab577&role.id=692ec53a198846cda9892422cde67a24 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:05.646186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207a68-1781-4309-b396-664f382093dd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:05.647220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207a68-1781-4309-b396-664f382093dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:05.647729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207a68-1781-4309-b396-664f382093dd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:05.649761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/456] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:05.655012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:05.655849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:05.656323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:05.683241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:05.683839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e34-dcc8-4215-b8e7-0ee2d8183dfc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:05.684120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e34-dcc8-4215-b8e7-0ee2d8183dfc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:05.684394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53e34-dcc8-4215-b8e7-0ee2d8183dfc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:05.685553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:05.686045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/457] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:05.691183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:05.691411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:05.691892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:05.720210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:05.725327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:05.809017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/458] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:05.828601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/459] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:56:05.845513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:05.854462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:05.876741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:05.964643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.032927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8404b16-de8b-427f-8d6a-488f8a476b27 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.033440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113644e-51f2-4f9e-b401-af526764b492 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.070205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.070567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.071012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.071263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.071360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.071544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.071637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.071816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.082798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.084140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.084246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.085659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-21e3d1b9-a47d-4629-9308-9839e149f4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:06.102776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:06.103663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:06.104967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:06.104967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:06.104967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.104967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:06.104967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/460] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:06.107826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:06.111709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:06.112952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:06.112952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:06.112952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.112952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:06.113210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597237d1-bd10-48a0-aff5-1668a2093a6c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.113210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597237d1-bd10-48a0-aff5-1668a2093a6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.113210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597237d1-bd10-48a0-aff5-1668a2093a6c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.113381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/461] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:06.114515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.116750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/462] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:06 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:06.117582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597237d1-bd10-48a0-aff5-1668a2093a6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.190998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.231955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597237d1-bd10-48a0-aff5-1668a2093a6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.235370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fa132-6ea1-4896-8b06-7d07c719f7a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.250615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.271143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.272136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.272619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.273090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.273640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.274215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:06.274319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.274592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.274868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.275157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.275613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:06.275921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.277253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c6f4-208f-4eec-8e8a-9103541b7936 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.277387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.290820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/463] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:06.292672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.293129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.293576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.296574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/464] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:06.298591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.316719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:06.317940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:06.386636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/465] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:06.387281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.421981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7ea0e-bdb5-4598-a0a1-426fbbae0703 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.439606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.461106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.461790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.462074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.462354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.463549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.465614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea7351-f88c-4845-b17f-1dcc250ea0c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.489874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeff5c5-2d2f-4e80-9fd6-b8f4af90a7d5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.492305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/466] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:06 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:06.493630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83550d4-0675-4d35-bdfb-d9b4218f8709 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.493921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83550d4-0675-4d35-bdfb-d9b4218f8709 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.494196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83550d4-0675-4d35-bdfb-d9b4218f8709 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.499775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/467] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:06.499775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83550d4-0675-4d35-bdfb-d9b4218f8709 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.506297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.550864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.551993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.552501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.553810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.554831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.557657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdccf17-d37f-481a-a5e0-7ba44fad530e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.588013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/468] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:06 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:06.590668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.591176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.591651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.614731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:06.616884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:06.633937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83550d4-0675-4d35-bdfb-d9b4218f8709 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.712768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.713697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.713982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.714309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.729258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.731120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.733313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/469] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:06.752114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:06.753022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:06.753986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:06.753986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:06.753986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.753986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:06.753986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/470] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:06.759331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.760391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.804336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.887701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:06.904571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4fc0-1991-417a-abaf-04656d43bbd1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.908295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e571790-73d2-4dd3-83d3-5dd143e27291 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:06.946768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.947579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.947864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.948140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.952179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:06.953104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:06.953571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:06.953996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:06.954865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:06.955851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.958248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.959938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:06.961141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:06.974497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/471] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:06.980053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:06.980664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:06.981654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:06.982694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:06.982694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:06.982694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:06.982694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:06.982694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/472] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:06.988009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:07.122430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:07.147423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074b13-5b11-4d61-81c8-d3ff9ac008f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:07.151586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9eafe-73fc-479d-a97e-83920a4b72f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:07.207441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:07.208129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.208429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.208716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.209435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:07.210895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d543ae-df3c-4539-ab98-1621ec574fb7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:07.221312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:07.222066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.222347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.222621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.223356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:07.223661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:07.225078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:07.242737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/473] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:07.248286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:07.259749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/474] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:06 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 02:56:07.260652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:07.338729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc10c2-7b81-43f6-8dc4-9cf79f768233 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.339062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc10c2-7b81-43f6-8dc4-9cf79f768233 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.339290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc10c2-7b81-43f6-8dc4-9cf79f768233 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.341700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/475] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:07.350003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.350354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.350556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.379052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483973-e3ea-4d2b-b1d1-a7b5d7abb040 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:07.389699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:07.391305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:07.425600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:07.426669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.427170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.427704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.428711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:07.430399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:07.477369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/476] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:07 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:07.529228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/477] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:07.554772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:07.627016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:07.684702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627dc65-5a0c-4301-b65e-cd86589fa57d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:07.721883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:07.722744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.723040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.723374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.724133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:07.725548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:07.732853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 02:56:07.732853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 02:56:07.732853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [parameters: {'id': 'b6bf9b2e86794bea8e07139e61ef1581', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 02:56:07.732853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 21 02:56:07.749586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:56:07.751305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 02:56:07.752187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 02:56:07.753185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:56:07.755868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:56:07.757849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': 'b6bf9b2e86794bea8e07139e61ef1581', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:07.759742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 02:56:07.766612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:07.769106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/478] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jun 21 02:56:07 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:07.769248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:07.842429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:07.874710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36587cdb-15d1-4015-b824-43e80e6fabc9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:07.912197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:07.913082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:07.913365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:07.913647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:07.925910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-70166c82-623a-4d5c-b34f-f3b47302f4a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:07.934116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:07.936216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 02:56:07.938173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:07.944651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/479] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:07.949547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:08.058388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b6a2-d736-4d5d-a7ce-3b5690502fd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:08.085026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:08.097576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:08.098334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:08.098623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:08.098937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:08.099810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:08.100129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:08.101553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a935135-9f3f-480a-9759-5aaac51daa42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:08.114863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/480] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:08.891036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ca97e-efc6-409e-ad15-2ade4970fca4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:08.891570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ca97e-efc6-409e-ad15-2ade4970fca4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:08.891570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ca97e-efc6-409e-ad15-2ade4970fca4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:08.893131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/481] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:08.899129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298878f4-3596-4f46-9d32-2a50ae23d89e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:08.900003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298878f4-3596-4f46-9d32-2a50ae23d89e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:08.900522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298878f4-3596-4f46-9d32-2a50ae23d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:08.918741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-298878f4-3596-4f46-9d32-2a50ae23d89e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:08.920274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-298878f4-3596-4f46-9d32-2a50ae23d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:08.990604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/482] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:09.014205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ca97e-efc6-409e-ad15-2ade4970fca4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.152383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ca97e-efc6-409e-ad15-2ade4970fca4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:09.192977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:09.193692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cb71-1659-4003-a8e1-a455b633844e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.194050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cb71-1659-4003-a8e1-a455b633844e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.194635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893cb71-1659-4003-a8e1-a455b633844e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.196006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/483] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:09.198495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:09.199298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.199584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.199865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.201845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.202202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.202548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.211186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:09.219081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:09.220118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:09.221129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:09.222833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/484] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:09.234169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.239393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:09.240701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:09.308708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/485] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:09.334333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.416875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:09.442829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3da23-92ea-45e8-997b-45289c0d868a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:09.497474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a7bfe-ebc6-4d20-ac50-5ceda7206886 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:09.532907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:09.533595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.533874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.534146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.535025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:09.535333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:09.536766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:09.541107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:09.541919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.542198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.542470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.557248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/486] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:09.560920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:09.562777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:09.578722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:09.579840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:09.580872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:09.583133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/487] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:09.593548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.683269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:09.704386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea371c-5b96-475b-b218-3915ce5b4a11 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:09.712264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:09.744630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:09.745555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.745978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.746410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.747322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:09.748578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:09.750470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:09.765295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/488] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:09.771241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.786514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584846e9-e942-4a18-ae81-fd1aa7ac31ca None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:09.855507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:09.856333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:09.856641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:09.856914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:09.880591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:09.882089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:09.912082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:09.913499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:09.914332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:09.914332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:09.914332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:09.914332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:09.914845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/489] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:09.923387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:09.976436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054b98d-ca5d-4e91-a889-8b95ed69e494 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.040006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.041007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.041294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.041570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.065906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.067409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.081429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869be0f7-9012-42d6-9672-254fd19d6021 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.088628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:10.093473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:10.094757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:10.095680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:10.095680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:10.095680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:10.095680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:10.099453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/490] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:10.101915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.118443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.119207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.119497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.119794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.120627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:10.120933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.122419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7a797-a1be-420e-8f51-84209c44640d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.126551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.136576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/491] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:10.137890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b069-66ef-4976-8e1f-3146f9abff2a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.138175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b069-66ef-4976-8e1f-3146f9abff2a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.138451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b069-66ef-4976-8e1f-3146f9abff2a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.139904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/492] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:10.140990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d35417-afd8-4460-a561-f2d881baab04 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.141270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d35417-afd8-4460-a561-f2d881baab04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.141545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d35417-afd8-4460-a561-f2d881baab04 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.142962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/493] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:10.144925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.145337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.145679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.162417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:10.163624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:10.213926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5c2-a38c-41be-a2d7-b746abee4e01 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.238148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/494] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:10.238607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.256456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.257689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.258385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.258866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.260266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:10.260668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.262165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037da340-cb63-4cc8-a830-9fbdf2a86d69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.276705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/495] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:10.281096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.281096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.281096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.296784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:10.297997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:10.347993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.354181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf45a2-d19a-4567-bf00-21dcdd759afc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.370559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/496] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:10.371483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.384305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.408954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.409841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.410175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.410451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.419337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.420888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:10.437838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:10.438768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:10.439656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/497] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:10.440214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.487916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73b8c7-be37-4cb0-9cb4-bf39e1a75a97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.528340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.529245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.529528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.529800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.541910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:10.549721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:10.550797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:10.551626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:10.552296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a27e7-9733-405e-8d4e-0cf89b1c58c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.552368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/498] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:10.553202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.599865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.600751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.601118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.601394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.609397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.610933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:10.626767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:10.627678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:10.628640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/499] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:10.629220 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.634442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.648522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.667824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267fe02-02e1-4cd5-8172-5c3c4678ceca None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.702767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.703578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.703865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.704137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.714818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:10.722657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:10.723543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:10.724341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/500] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:10.725422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.739591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428390cb-c1db-483a-8c0e-325642b475ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.778336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.779077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.779314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.779588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.780471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:10.780788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.782265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.819074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/501] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:10.824027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.835137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f0c4e-7c7f-4e1f-9fdc-fdf80824ead0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.864523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.871091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.871770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.872057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.872336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.873122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:10.873431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.874835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.883057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/502] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:10.883837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:10.901154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:10.926356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254abe72-179b-4208-b235-a12d464af6c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.959761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:10.960471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.960778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.961044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.961889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:10.962199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:10.963641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562291e-d6be-4ea6-b41f-96b8f3830ded None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:10.980150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/503] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:10.981429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45451c68-7872-4638-b362-f6041b436b02 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:10.981713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45451c68-7872-4638-b362-f6041b436b02 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:10.981986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45451c68-7872-4638-b362-f6041b436b02 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:10.982077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017651d9-aa65-4c8f-aa2c-ebe8cb53ee53 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:10.983351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/504] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:10.988009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45451c68-7872-4638-b362-f6041b436b02 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.015904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.016608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.016890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.017171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.017916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:11.018224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.019643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.027601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/505] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:11.028372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.084369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.094269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45451c68-7872-4638-b362-f6041b436b02 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.125914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.130556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ed71-7dd0-4aef-9ff5-4aef06c77945 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.134812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.135459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.135739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.136008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.164367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.165163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.165430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.165712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.176485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.177944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:11.194589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:11.195463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:11.196358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:11.196358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:11.196358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.196358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:11.196358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/506] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:11.196682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.212979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.214579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc987b5-7675-4daa-8cce-b1325f0267b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.304525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539aeca5-78cf-419d-88f8-799da0458bc8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.311966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.318227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/507] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:11.318227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.318227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.318227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.331956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:11.333181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:11.340994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.341594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.341883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.342171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.358621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.399897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/508] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:11.400907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.412803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.414406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.510035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e200e1-0183-4c35-9424-87f97bffe80e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.511201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/509] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:11.512037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.520072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.550417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.551397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.551678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.551949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.562444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.563925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:11.581552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:11.582459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:11.583301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:11.583301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:11.583301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.583301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:11.583301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/510] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:11.583603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.599159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.613204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422f8c-72a9-489d-b8f4-2259ee465c22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.646939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.647606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.647929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.648190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.648946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:11.649241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.650614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.662048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/511] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:11.662843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.686044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98c3a-5270-42a1-b912-2891791349b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.722324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.723122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.723405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.723676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.726971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.734354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.735848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:11.753528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:11.754458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:11.755293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:11.755293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:11.755293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.755293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:11.755293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/512] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:11.755556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.763591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154adfe6-64c3-457e-bb1a-4ca1b3def2df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.796007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.796837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.797120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.797395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.807683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.809183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.823428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:11.826098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:11.826932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:11.827767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:11.827767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:11.827767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.827767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:11.827767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/513] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:11.828152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.860640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b721f7c-9d32-486a-b1bf-b1b0f3346ad2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.896983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.897984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.898290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.898594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.909968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.911435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:11.929021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:11.930323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/514] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:11.931175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:11.945330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e782c5e-d053-498d-92e4-44471f878ab7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:11.962457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:11.980270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:11.980978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:11.981260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:11.981535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:11.982260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:11.982786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:11.983952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:11.995508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/515] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:11.996342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.036935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447e43-dca7-4b67-8ae5-d6467b666dd0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.054634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.072610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.073425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.073711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.073992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.081762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.083258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.097494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78646e-13d2-459b-9d9e-174621a9884a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:12.097636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:12.098490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:12.099388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/516] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:12.099932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.134647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.135215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.135492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.135763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.136513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:12.136825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.138247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.149686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/517] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:12.150840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.192941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.205813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee1b0-5c2f-428f-936d-4b27a56872aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.240754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.241450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.241735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.242012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.242751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:12.243067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.244496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.256233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091e27d-791d-4ee1-ae9a-88cc9645dc18 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.257305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/518] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:12.262654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.289996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.290614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.290911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.291191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.291921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:12.292228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.293658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.304679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.305566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/519] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:12.306377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.369362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b152c7-f0fe-409e-8c88-0df073712e45 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.405179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.405370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.406241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.406496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.406790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.409319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515af1-8b6d-432e-b4d8-f6f28bf4fd41 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.414503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.416001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:12.431318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:12.432281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:12.433218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/520] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:12.433777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.444761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.445440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.445716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.445988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.446834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:12.447145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.448609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.485517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/521] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:12.486372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.533439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.537270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4a180-aaf3-4439-91bd-a5e348a1f54d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.570022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.570838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.571115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.571383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.577878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.579357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.592216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20376a-6501-498e-a475-c278748622d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:12.593366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:12.594229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:12.595070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/522] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:12.595525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.630302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.630980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.631262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.631541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.632248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.633741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.656177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/523] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:12 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:12.657234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.685673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.701150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68adf5-2c55-41a4-a669-47dad96f5d23 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.739789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.740647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.740931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.741211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.749420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.751301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:12.767662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:12.768723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:12.769594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:12.774030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/524] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:12.774801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.793883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97ef53-247e-416d-a0bf-7386a2ce8d56 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.797228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.834797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.835486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.835774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.836052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.836956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:12.837301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:12.838727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:12.861513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/525] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:12.862460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:12.895217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea24b-76b3-4dc9-9b01-13da98ebf19b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:12.934482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:12.935134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:12.935364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:12.935618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:12.942468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:12.974854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb534e2-5ee4-4810-841f-2eff2d9ac334 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.008785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.009442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.009722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.009995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.010783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:13.011089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.011514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.012577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.013032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.035603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/526] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:13.035603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.038066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.132398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/527] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:13.132398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.143995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011224ad-e8cc-4939-8d8e-e6338bce4906 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.179973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.187176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.187797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.188083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.188363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.189193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:13.189513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.191124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.202840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/528] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:13.203654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.254318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90cb2d-2b38-4d99-b4a1-9574383fbb57 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.275155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.294209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.294866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.295149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.295426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.310207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e13cd-bed9-44bf-b262-799781098299 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.346646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.347414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.347699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.347981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.348883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:13.349160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.350605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.364458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/529] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:13.364458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.375035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.377214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.435862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.489535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee47d-f957-4997-b397-120bc5c1b72c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.492892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/530] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:13.493917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.514922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.537291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.538570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.539005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.539461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.557456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.559838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:13.583967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:13.585098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:13.586153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:13.586153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:13.586153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:13.586153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:13.593134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/531] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:13.593598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.610700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa8880-bfcc-4a31-b5d4-93adeff3c2bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.647839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.648657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.648943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.649231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.650214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=9bdf485ec3ca4f3785bb47f3706c4466) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:13.650527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.652103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.664757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/532] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/role_assignments?user.id=9bdf485ec3ca4f3785bb47f3706c4466&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:13.665920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.748529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa7117-84e2-4930-88be-5bf999b80ee7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.758170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.791382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.792856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.793928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.794494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.809666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.811598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:13.836968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:13.837923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:13.838758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:13.838758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:13.838758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:13.838758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:13.841722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/533] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:13.850195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.859688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:13.892449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68255e4-9a5a-46be-be31-dd55f41382a7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:13.929609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:13.930287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:13.930574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:13.930851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:13.931593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:13.931904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:13.933489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:13.949931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/534] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:13.951170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:13.974204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa27d4-47d5-4e3c-953a-a646fe082f1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.018173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.018970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.019458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.019743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.020502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:14.020814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.022196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.052921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/535] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:14.054004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:14.059098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd2db-d986-4173-a456-936ef7df91cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.100478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:14.101461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.102278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.102562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.102842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.113296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.114836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:14.140250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:14.141224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:14.142077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:14.147904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/536] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:14.147904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:14.156905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:14.250822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67899fc3-e0e9-4145-b312-759818f280e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.270112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921fe94-0aea-481a-8e3e-53ee4fa21067 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.286630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.287458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.287844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.288231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.289215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:14.289624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.291185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.316903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.317711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.317998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.318279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.319865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/537] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:14.321917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:14.325355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.326836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:14.355886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:14.357039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:14.358126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/538] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:14.365386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:14.400837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:14.431926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd17ff-292c-4a01-bf48-28116a44c616 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.467051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.467919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.468300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.468704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.469814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=18bc17792b4d4879a1fd533206245694) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:14.470227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.471986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.484701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/539] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:14 2022] GET /identity/v3/role_assignments?user.id=18bc17792b4d4879a1fd533206245694&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:14.507695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:14.541067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a59e7-08a5-4228-a634-01729597046b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.579443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.580123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.580423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.580714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.581662 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:14.581975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.583467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df2171-4d1a-4e02-9639-a745a6f81aa4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.597425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/540] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:14.603008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:14.738833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215b61e-f861-4d01-b314-1a1a85a6f319 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:14.781267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:14.795284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:14.795882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.796229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.796535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.797434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=357dd2446e9248cab4e559dc5ce931b1) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:14.797750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:14.799211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffa26d-05a9-48af-88b7-80d626220dfb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:14.808808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/541] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:14 2022] GET /identity/v3/role_assignments?user.id=357dd2446e9248cab4e559dc5ce931b1&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:14.898361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481fae-b940-4b75-98af-ec5afdc8fd64 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.898712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481fae-b940-4b75-98af-ec5afdc8fd64 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.898939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e481fae-b940-4b75-98af-ec5afdc8fd64 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.900697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/542] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:14.908260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:14.908586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:14.908856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:14.937741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:14.939021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:15.033236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/543] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:15.057494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e481fae-b940-4b75-98af-ec5afdc8fd64 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:15.196893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e481fae-b940-4b75-98af-ec5afdc8fd64 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:15.242878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:15.243694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.243976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.244253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.256035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:15.264042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:15.272242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:15.273178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:15.273961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/544] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:15.279377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:15.416511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e369a-c1d8-4da2-92e6-4a5f5af85a6c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:15.442238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:15.458366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:15.459043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.459327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.459602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.460439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:15.460772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:15.462223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:15.471180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/545] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:15.477956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:15.607498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fb5cb-e730-4184-8171-f73a6ef04292 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:15.645641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:15.646728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.647154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.647568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.661296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:15.663032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3bb6a41d-ad7c-43d4-b96f-a333997ee47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:15.685290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:15.686339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:15.687379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:15.687379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:15.687379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:15.687379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:15.689005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/546] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:15.694848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:15.829187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf4971-1ca7-4b72-b75b-178f499f5071 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.829849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf4971-1ca7-4b72-b75b-178f499f5071 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.830319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf4971-1ca7-4b72-b75b-178f499f5071 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.832289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/547] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:15.837223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.837735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.838525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.854031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2dc6d-45bc-43f4-9874-b32f2a0d37b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:15.858665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:15.860091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:15.917334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:15.918016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:15.918301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:15.918586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:15.919325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:15.919636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:15.921333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:15.934559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/548] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:15.940264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:15.947407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:15.973211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/549] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:16.007720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.018753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.098108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8bc66-fa3f-41bb-8989-2fae799e2268 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.140772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.142031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.142538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.143188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.156574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:16.161655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:16.190602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:16.192164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:16.193268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:16.195587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/550] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:16.199561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.202411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5894f95-cfbf-4374-ba95-3883b340405b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.285498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.287103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.287702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.288211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.312635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda594b-1840-4de0-801b-8590a493df8f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.314349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.325395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:16.334207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:16.335194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:16.335995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/551] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:16.341421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.368071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.369206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.369912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.371525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.372847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:16.374037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:16.376092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d6a54-145e-49d3-8a45-b9f89bd3d2c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:16.405866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/552] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:16.407436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006702c-3034-4143-9204-32f66c554700 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.407947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006702c-3034-4143-9204-32f66c554700 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.408308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006702c-3034-4143-9204-32f66c554700 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.415151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/553] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:16.416385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.416520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006702c-3034-4143-9204-32f66c554700 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.502773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a02d-df40-41c2-9be4-5d25d803c902 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.553667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.554864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.555297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.555719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.556813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:16.557136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:16.558566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d405b0-7cf8-4b2e-b1fd-0259b39cd544 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:16.560882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006702c-3034-4143-9204-32f66c554700 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.576771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/554] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:16.576771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.576771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.576771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.600232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:16.602324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:16.608970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.609690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.609972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.610298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.648353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.701209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:16.702628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/555] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:16.702746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:16.703791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.760037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.829339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/556] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:16.830213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.857940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6931472-661b-4c49-80b0-9efc037273f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.897404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:16.901159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.903594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.903887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.904171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:16.919882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:16.921683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:16.943838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:16.944992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:16.945926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:16.945926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:16.945926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:16.945926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:16.945926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/557] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:16.946638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65153d57-84d0-4d5a-a842-916a99af77f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:16.947079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:16.986680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:16.988260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:16.988770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:16.989235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.001969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:17.011037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:17.013558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:17.020862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/558] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.022203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.054439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.074800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e565-1f86-4e3d-9c01-3c18deaee93c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.120301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.121158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.121453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.121726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.134967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.136491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.136787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c88414-b260-4e86-8eac-d0b62842c76a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.166065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:17.166610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.167460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.168452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:17.168452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:17.168452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:17.168452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:17.172766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/559] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.174171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.180853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.181525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.181797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.182081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.182803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:17.183111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.184549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741a14e-1964-427e-919c-78edcf14b0a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.197082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/560] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:17.198409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c9e3-f0de-4d66-9551-acac839864b3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.198696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c9e3-f0de-4d66-9551-acac839864b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.198968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c9e3-f0de-4d66-9551-acac839864b3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.203472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/561] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:17.203472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3c9e3-f0de-4d66-9551-acac839864b3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.311222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbc3e8-21f7-4ba3-8df5-857aca016c00 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.337430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3c9e3-f0de-4d66-9551-acac839864b3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.347886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.348605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.348893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.349174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.349944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:17.350253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.351752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.354601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.361606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/562] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:17.362483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.381612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.382268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.382561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.382833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.383550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:17.383862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.385290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54afb23a-ea3e-44f1-a516-557996b52cfb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.397488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/563] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:17.398890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339024e-ea26-4a68-b289-b4738af3ee65 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.399170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339024e-ea26-4a68-b289-b4738af3ee65 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.399442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339024e-ea26-4a68-b289-b4738af3ee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.421284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a339024e-ea26-4a68-b289-b4738af3ee65 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:17.422498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a339024e-ea26-4a68-b289-b4738af3ee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:17.460638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.474664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24f98-9ac4-49d7-94a7-a211b1b650c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.497120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/564] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:17.498609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704cd46-b911-437b-84ff-d4d0ebbc7d1a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.498963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704cd46-b911-437b-84ff-d4d0ebbc7d1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.499314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704cd46-b911-437b-84ff-d4d0ebbc7d1a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.501190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/565] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:17.505922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704cd46-b911-437b-84ff-d4d0ebbc7d1a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.511874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.512774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.513127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.513467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.521957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.523448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:17.539295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.540488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:17.541682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:17.544556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/566] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.545514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.603899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.621784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704cd46-b911-437b-84ff-d4d0ebbc7d1a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.650826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055fbd7-2401-46be-bab2-4ab44d65c605 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.657742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.658534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.658851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.659120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.670144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.671614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.686372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.687171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.687456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.687734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-48479678-0e0a-4534-855c-135cdd7d9c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:17.689804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.690920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.691911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:17.691911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:17.691911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:17.691911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:17.691911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/567] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.693541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.693833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.694130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.695115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.696724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.710743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:17.711933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:17.712101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:17.714200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/568] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.714782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.731751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.781240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/569] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:17.782157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.813454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256a7b2-ede0-4579-a266-33b65a61ac51 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.830251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.847644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.848479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.848773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.849050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.859150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:17.866820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:17.867889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:17.868932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:17.873029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/570] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:17.873900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.894800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9cfd59-12bb-44a9-b419-747ad38d6d8a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:17.928390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:17.929125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:17.929399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:17.929667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:17.930484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:17.930781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:17.932224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:17.949406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/571] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:17.950202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:17.961970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:17.979106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea78c12-f9d3-4ff7-9610-fb0f57987e93 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.012556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.013222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.013507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.013784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.014508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.014820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.016200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.027943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/572] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.028861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.055091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cd3bd-69ce-4b18-ae04-65d595d83920 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.068272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.094505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.095176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.095482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.095735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.096590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.096909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.098379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.109910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/573] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.110647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.129521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec61a0-ba12-4a8d-bf8e-b3bb08ffc509 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.165236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.165488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.165789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.166061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.176997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:18.184792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:18.185893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:18.186718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/574] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:18.187404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.193311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.225614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a202df-e438-4d74-9dc9-4a6bba137d41 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.259404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.260173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.260493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.260778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.261535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.261847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.263277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.271971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/575] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.272796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.286428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.291118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b85e-1ed1-4d5e-b468-93dce7c536a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.324747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.325660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.325947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.326221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.337428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.338885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:18.355517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:18.356526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:18.357454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:18.357454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:18.357454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.357454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:18.357454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/576] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:18.357725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.381297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3991392-cdf4-402d-823d-0ca394e876d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.410761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.416337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.417219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.417502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.417776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.425257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.426885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:18.440936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:18.441900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:18.443387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/577] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:18.447459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.456881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2088a3-0fcc-42c4-8a96-9f685ac3d288 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.490100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.490733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.491018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.491300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.492133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=5b4f45c15c2949f5bd0e606b3cf00493) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.492476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.493947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.499331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/578] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/role_assignments?user.id=5b4f45c15c2949f5bd0e606b3cf00493&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.500100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.536737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.553250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26688de6-1e22-4b7e-b190-23e8ad187591 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.591537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.592231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.592546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.592826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.593601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.593914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.595355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.604684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/579] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.605566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.646386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf19e85-ccef-4eaa-9825-db1829a6fe6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.671160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.684165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.684999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.685284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.685557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.697239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.698694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:18.715618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:18.716553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:18.717456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:18.717456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:18.717456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.717456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:18.717671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f401-9ed1-4b7e-a9a4-047e11a76873 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.721519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/580] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:18.722366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.752655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.753802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.754230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.754509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.755232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.755542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.756963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.773518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/581] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.773518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.792883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.833681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6272556-e567-4278-bb76-c4076ffa9e86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.867984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.868708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.869012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.869306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.870166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:18.870475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.871916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.873024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d778-3eda-4b16-bcac-969594f1fe7c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:18.889822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:18.894803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/582] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:18.894803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:18.918797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:18.919526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:18.919808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:18.920080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:18.930899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:18.932368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:18.950644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:18.951621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:18.952554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:18.952554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:18.952554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:18.952554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:18.952554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/583] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:18.957155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.001376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfd625-6808-4bf0-bda7-3f75d5cc1719 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.026315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.040434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.041104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.041386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.041662 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.042397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:19.042703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.044082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.059221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfe90e-70bd-4820-a956-d40c0af2e261 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.060676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/584] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:19.062627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.099234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.100829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.101116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.101392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.109900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.111975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:19.130455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:19.131388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:19.132283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:19.133482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/585] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:19.134443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.168676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.174371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b960a-745f-45cb-a535-8d32141d47c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.209555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.210470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.210786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.211097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.221905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.223457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:19.241368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:19.242300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:19.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:19.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:19.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:19.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:19.243218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/586] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:19.243501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.249712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7f734-d474-4ff4-8a77-2e30330f1332 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.283011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.285992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.286797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.287083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.287358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.288085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:19.288553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.290187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.302572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/587] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:19.303200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.354964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db732d2-8fc9-4823-88f4-7288722888a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.390106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.390907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.391193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.391469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.393013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.399587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.401084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.408566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3b697-04fd-49dd-9882-a59b21c89c2e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:19.415868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:19.416845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:19.417788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:19.421147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/588] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:19.422530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.445318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.445915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.446215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.446476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.447266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:19.447575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.449060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.465389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/589] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:19.465389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.533996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.541064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a681c6-1207-494e-927e-d9da04f09465 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.567806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0e01-d983-44f9-b311-013eb8db6967 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.575019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.575718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.576006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.576284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.577074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:19.577390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.578818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.596810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/590] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:19.596810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.609168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.609986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.610268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.610541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.618563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.620017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:19.634603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:19.635531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:19.636563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:19.640436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/591] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:19.641260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.650253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.702792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eed0-9683-4f4f-8ebc-37c8960b35dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.738828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.739466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.739755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.740041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.740938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:19.741252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.742748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.747833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bb646-685b-4491-954b-631f9a3d27da None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.763978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/592] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:19.764828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.782864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.791431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.792348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.792665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.792947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.808440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=1c16671fd5844a77ac6e56dff36f86f8, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.809917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.832449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/593] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:19 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:19.832449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.881852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb246d5a-8906-4f48-a467-ff1501de3233 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:19.904622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:19.930264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:19.931081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:19.931359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:19.931639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:19.939491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:19.941215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:19.956448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:19.957393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:19.958297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:19.962561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/594] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:19.963698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:19.968124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78acb4-0a56-45d3-8d69-7aa4332cc7db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.018246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.019271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.019755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.020041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.020949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:20.021270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.022753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.047638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/595] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:20.048544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.059780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.088932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae0e65-59f2-49f7-b320-1bea195b08a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.126411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.127437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.127817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.128344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.140306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.141901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:20.161403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:20.162363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:20.163256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:20.163256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:20.163256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:20.163256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:20.163256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/596] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:20.168225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.189452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad33b9-3971-4711-93df-adfcd74a74b3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.212246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.244942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.246267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.246989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.247471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.249359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:20.249896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.252359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.271468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/597] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:20.273001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.280234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995fce1-0b5a-464f-b28b-0b41ede0e81c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.307665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.323743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.324588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.324868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.325137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.337009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.338577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:20.357635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:20.360140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/598] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:20.361751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.412543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321f311-d8ea-4d70-ba2d-79a44e42621d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.454342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.455516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.456487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.456974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.457912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:20.458517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.460244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.467900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8b4b-e24a-4c82-b8b2-1ee2d4ac4eba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.474728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/599] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:20.475925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.508602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.509531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.509849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.510130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/9bdf485ec3ca4f3785bb47f3706c4466/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.515630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.526722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=9bdf485ec3ca4f3785bb47f3706c4466, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.528183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.542676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/600] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:20 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/9bdf485ec3ca4f3785bb47f3706c4466/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:20.543506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.584130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.614620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806537b7-2755-448f-ac06-5e98b97b1503 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.654719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.655405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.656174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.656174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.656754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:20.657063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.659932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c97445-f619-4034-8c23-774712f631a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.678380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/601] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:20.679943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ca3d-53c8-4862-9806-b00761178a53 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.680237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ca3d-53c8-4862-9806-b00761178a53 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.680562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ca3d-53c8-4862-9806-b00761178a53 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.682243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/602] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:20.682996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ca3d-53c8-4862-9806-b00761178a53 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.698703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255de96-182b-4919-8063-d0bf5eef98e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.744579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.746212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.746926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.747356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.760243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.762325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6937561f-b5aa-428b-a09c-b87fc841d1be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:20.780147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:20.781106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:20.781994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/603] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:20.782923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.782923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.783215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.800587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:20.801833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:20.807446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:20.816667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ca3d-53c8-4862-9806-b00761178a53 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:20.860814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:20.862197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:20.862755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:20.863039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:20.872712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:20.874186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:20.876500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/604] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:20.877317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:20.900462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:20.901421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:20.902319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/605] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:20.909273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:20.913100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.004789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213ec8b-0c6d-4b69-b96a-047bf9854db8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.051244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d51dfd-f1e9-436e-aa33-ce1be3e4b93e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.052402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.053198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.053826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.054179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.056056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:21.056458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.058002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.074894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/606] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:21.076730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.101096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.101880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.102237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.102584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.104011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:21.104387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.106268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.128445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/607] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:21.128445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.147850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.177888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.200866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ce5f5-555a-42c6-b1ce-38dffc22380f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.236144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.236747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.237031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.237401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.248157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:21.256623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:21.257547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:21.258470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/608] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:21.259155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.259858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a3c11-4558-49cc-9a83-d17bd5d525b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.296036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.297346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.297833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.298302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/18bc17792b4d4879a1fd533206245694/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.314189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=18bc17792b4d4879a1fd533206245694, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.316024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.333891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/609] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:21 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/18bc17792b4d4879a1fd533206245694/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:21.335146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.365599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424849-7e8d-41e1-9290-6e1e2a46588e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.401635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.402570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.402833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.403208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/357dd2446e9248cab4e559dc5ce931b1/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.412836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.414920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.421314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=357dd2446e9248cab4e559dc5ce931b1, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.423648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.452019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/610] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:21 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/357dd2446e9248cab4e559dc5ce931b1/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:21.464178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72315d-3049-41d1-a60a-24f3e9d41b26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.506321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.507453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.507917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.508367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.509365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:21.509863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.511504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.522471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/611] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:21.528273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.667620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4919e60-1fb5-4edf-bbb5-01a0475a097a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.705510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.706358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.706641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.706917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.717656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.719167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:21.738708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:21.739634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:21.740531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:21.740531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:21.740531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:21.740531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:21.740531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/612] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:21.745750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.867994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd627e-f4b8-47be-b392-f979dca88640 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:21.898960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:21.908867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:21.909866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:21.910371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:21.910868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:21.911799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:21.912304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:21.913939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cb61c-0831-46f6-afc2-c12dfed58bc0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:21.927410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/613] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:21.932757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:21.998996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.068224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d1cc1-ffff-45bc-9264-02df9f736ead None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.116394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.117232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.117515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.117827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.133633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.135113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.137469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a0-ce3b-4edf-84c5-65beb9dd5dd0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.137770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a0-ce3b-4edf-84c5-65beb9dd5dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.138057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af89a0-ce3b-4edf-84c5-65beb9dd5dd0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.144518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/614] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:22.148468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.148769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.149055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:22.152059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:22.152975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:22.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:22.156163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/615] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:22.167090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.185493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:22.186722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:22.303631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/616] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:22.320305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.323180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e62e-3316-4583-9724-84fed4ce08c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.351235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.360750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.361671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.362192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.362663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.363704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:22.364218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.365865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.400073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/617] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:22.413885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.475218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8140c1-f9fc-4b8b-b1ce-ab5e87adafd3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.493404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.513641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.514518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.514804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.515132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.526787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:22.535069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:22.535984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:22.536863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/618] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:22.549018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.580303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0ee2e-6334-4e76-809b-0066928828ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.624440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.625594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.625882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.626162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.636732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.638203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:22.657616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:22.659738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/619] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:22.668811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.671505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5da0fd-247a-4875-9567-693a8ad011a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.677093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.713190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.714283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.714770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.715245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.716333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:22.716879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.718562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.728798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/620] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:22.733782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.767607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a46e3-aa17-4215-9d99-dd252f2dba8f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.782337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.805398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.806218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.806338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.806619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.807432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:22.807653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.809065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.821264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/621] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:22.826634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.855002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eeb5d-1074-4561-a5ae-a565f42c00ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.893462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.895371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.895874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.896359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.897331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:22.911115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.913728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:22.933069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f314ee-5fc7-440a-8692-33a9783cf213 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:22.936282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:22.937229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:22.938151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:22.938151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:22.938151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:22.938151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:22.944650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/622] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:22.947395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:22.974100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:22.975298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:22.975795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:22.976742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:22.994619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:22.997809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-70e8cae6-b08f-4059-af89-84398e1bcb98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:23.032960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:23.034001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:23.035183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:23.040635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/623] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:23.042229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3a63-7ffc-4e27-9fb1-2b9231eb0fd5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.042735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3a63-7ffc-4e27-9fb1-2b9231eb0fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.043235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3a63-7ffc-4e27-9fb1-2b9231eb0fd5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.045860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/624] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:23.046906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3a63-7ffc-4e27-9fb1-2b9231eb0fd5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.087960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.099835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ef85-da5a-425b-949e-01fd86162aeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.135768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.136525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.136792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.137078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.137795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:23.138100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.139690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b0f93-aa82-4485-aeaa-261d64f48b88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.153416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/625] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:23.153711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.153988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.154501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.180203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:23.181474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:23.194353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3a63-7ffc-4e27-9fb1-2b9231eb0fd5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.225285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.231473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.232444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.233045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.233514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.234575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:23.235128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.236845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.255582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/626] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:23.255582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.264517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/627] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:23.265488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.357908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a521ca-7e31-4c73-a745-77ceaf80478b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.376798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.387846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b3966-7564-4998-a9b2-05c20edc5ae0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.394014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.395473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.396030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.396536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.404808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.406504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:23.421324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:23.422485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:23.423748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:23.431366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/628] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:23.431366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.431964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.432928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.433210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.433480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/5b4f45c15c2949f5bd0e606b3cf00493/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.448997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=5b4f45c15c2949f5bd0e606b3cf00493, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.450590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.472439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/629] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:23 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/5b4f45c15c2949f5bd0e606b3cf00493/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:23.472439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.520888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.641103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327225-f7a3-43e7-bd13-aedbd01c31d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.678613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a149f-f37b-4d6a-be96-b6c021785e96 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.686130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.686986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.687270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.687544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.699308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:23.708246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:23.709266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:23.710119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/630] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:23.715524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.718030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.725998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.727473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.727957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.728442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.729863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:23.730412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.732886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.753006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/631] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:23.758998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.827856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5069-5b92-4984-8fef-8cb0f3f4479f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.840440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.862766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7870-8651-43ab-b521-2f4c7dfbe479 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:23.868071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.868788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.869075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.869352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.870106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:23.870413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.871852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:23.882173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/632] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:23.887672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:23.902218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:23.903197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:23.903677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:23.904164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:23.968708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:23.982314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:23.984109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e7ab9-35e9-4b14-9f46-fbb736e01f7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.004127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e720-3640-47fc-adfe-01fefa9baf40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.048374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.050084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.050794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.051522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.065103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.067307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8acba722-8f6d-442b-b8d0-4a1ca53ee829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:24.090911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:24.091880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:24.092796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:24.092796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:24.092796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:24.092796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:24.093909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/633] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:24.098903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845579b-5eec-4baf-beaf-4e399929b8c1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.098903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845579b-5eec-4baf-beaf-4e399929b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.098903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845579b-5eec-4baf-beaf-4e399929b8c1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.100697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/634] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:24.102637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845579b-5eec-4baf-beaf-4e399929b8c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.107662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/635] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:56:24.110144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.110437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.110714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.130852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:24.132077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:24.142190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.228538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/636] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:24.228538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.247824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.251253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845579b-5eec-4baf-beaf-4e399929b8c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.286855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.287532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.287815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.288087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.289271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:24.289734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.291124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f3919-8fb9-4b03-89d8-472da8b10962 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.302864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/637] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:24.304463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535cbb4-0de1-48be-b7ac-e37e0d85146f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.304877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535cbb4-0de1-48be-b7ac-e37e0d85146f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.305156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535cbb4-0de1-48be-b7ac-e37e0d85146f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.306586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/638] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:24.307057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535cbb4-0de1-48be-b7ac-e37e0d85146f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.347993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82104b0c-ade4-4a16-9986-0202b31d4980 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.384706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.385611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.385901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.386180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.394005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.398181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.399686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.412997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535cbb4-0de1-48be-b7ac-e37e0d85146f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3a82daf2-ee99-4d67-98b5-69246e5258a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:24.423510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:24.424478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:24.425552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:24.425552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:24.425552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:24.425552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:24.425552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/639] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:24.426404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.426404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.426655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.443817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:24.445025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:24.455593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.456710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.457171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.457629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.469170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:24.477563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:24.478463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:24.479373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/640] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:24.480074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.513793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.519380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/641] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:24.520444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.597001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07fcfa-4aaa-4e91-a774-01b1a8cf2248 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.620367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d9303-eacb-435c-a945-b72881be6f94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.633338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.634532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.634839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.635114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.643099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.645116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.653661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.654331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.654608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.654876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.655607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:24.655914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.657367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:24.659975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:24.661003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/642] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:24.666729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.669107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.674720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/643] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:24.677791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.739048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.773437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b93b37-73eb-4f41-9dc2-9751cbcca8be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.787024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe64a6d-474e-4e9c-a3cc-941a6a2b3aa1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.810305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.811001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.811287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.811567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.812345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:24.812688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.814157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.823373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.823885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/644] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:24.824696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.824818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.825215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.825633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.839437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:24.847219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:24.848190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:24.849222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:24.853421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/645] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:24.854023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:24.902588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.941725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fda799-bb3d-44cf-b6fc-0eefd53ba2b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.954676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2344415-c600-4d33-a357-41aeceda5c40 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:24.975035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:24.980787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.981473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.981744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.982021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:24.982863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:24.983170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:24.984718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:24.991487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:24.992234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:24.992548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:24.992825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.000587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.002058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.008008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/646] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.008008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:25.016464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:25.017368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/647] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:25.023346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.116086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ac7a-8a56-4ae8-a821-144638ac72ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.122742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e769a-24e1-45e2-b29f-def2f61849aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.141008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.152315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.153613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.154443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.155334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.166401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.166620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.167300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.167589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.167867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.167974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.168660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:25.168974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.170615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.179495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/648] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.180237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:25.190554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:25.191496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/649] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:25.193002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.233401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.294094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93f57-f608-4e9a-a49b-a9428a68242d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.306647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e3ff-c804-4251-8b9a-583007e98796 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.329434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.330085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.330370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.330709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.342691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.343371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.343945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.344231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.345210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:25.345522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.347039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.358969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/650] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.359782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.397971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.410974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.413020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.457784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.464032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2946cb-4148-489b-84db-1c8088e224d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.498479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.499282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.499566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.499834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.512272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.514170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.522625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/651] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:25.522625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:25.536630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:25.537538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:25.538429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:25.538429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:25.538429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.538429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:25.542631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/652] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:25.543425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.622617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda282-1e65-4160-837d-eda4d6621483 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.635535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.648735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdedafa-607e-4c21-b88e-d9efa80f6254 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.660330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.661186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.661475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.661752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.662486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:25.662798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.664207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.677185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/653] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.678287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.683817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.684910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.685812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.686160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.686439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.687281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:25.687593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.689071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.704464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/654] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.704464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.809323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7b4bb-3d79-42bc-9677-df658161f269 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.815594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070b836-7ec7-4329-83b3-6049aa954a94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:25.845964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.847048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.847610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.847610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.856580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:25.857637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.858080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.858366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.859106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:25.859419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.860859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.861349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:25.862918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:25.880197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/655] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:25.881109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9bdcd035-71ca-4205-8a2b-07f626167ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:25.883084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:25.883983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:25.884980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:25.884980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:25.884980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:25.884980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:25.884980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/656] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:25.886015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a9d-08e0-44d7-8c2b-bdb75a471cf0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:25.886303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a9d-08e0-44d7-8c2b-bdb75a471cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:25.886579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3a9d-08e0-44d7-8c2b-bdb75a471cf0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:25.887845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/657] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:25.892775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f3a9d-08e0-44d7-8c2b-bdb75a471cf0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:25.900991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.959940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:25.999523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ce681-3b68-4270-8488-83741caa1079 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.012282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f3a9d-08e0-44d7-8c2b-bdb75a471cf0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.035922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.036837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.037133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.037416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.045034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.046483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.049381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.050526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.050819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.051102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.059824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:26.061299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:26.062204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:56:26.063142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:26.064393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.064493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/658] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:26.064493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-37e21782-663c-414c-937b-798cc5aabdad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:26.078631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:26.079827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 02:56:26.081555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:26.084450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/659] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:26.085743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.086310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.086310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.103764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:26.104982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:26.155027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.179431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/660] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:26.180740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.195431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf1d6-bc02-44ce-a888-616aa261626e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.234013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.236834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.237566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.237930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.238101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.238915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:26.239114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.240564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.258093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/661] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:26.258093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.289518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e21d0d-4bcc-4082-bf85-5d091b0d5eb7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.324278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.324982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.325268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.325539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.326377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:26.326685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.328117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.350208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/662] 10.222.0.16 () {56 vars in 1137 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:26.350208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.366334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2c4d2-7de1-4e39-b243-e4c39599be7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.397308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.403961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.404686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.404974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.405302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.406086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:26.406817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.408553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.426526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/663] 10.222.0.16 () {56 vars in 1139 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:26.429402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.455310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe57e0-81cf-4052-ab54-b179322c7fb2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.456967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.490377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.491186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.491493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.491767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.502595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:26.510316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:26.511242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:26.512063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/664] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:26.512796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.554668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd6dcb-673f-4fe7-811e-5325cd770c39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.594720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.596205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.596751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.597050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.604175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.605895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.613690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13047488-ea39-4d8f-8a2d-5377d39beeb8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:26.620669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:26.621537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:26.622440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:26.626374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/665] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:26.626374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.652170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.652865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.653150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.653425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.676482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.686729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.725691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7636a-dff9-43a3-88d1-43e9f9265231 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.727510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.729155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.759468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.760124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.760429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.760718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.830060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.831593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:26.836954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/666] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:26.837980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.886299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.905276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:26.943592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/667] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:26.944469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:26.952182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe58b45-5fb0-4a51-ba65-0bc3ed03b3e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:26.987285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:26.988240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:26.988752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:26.989241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:26.990077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:26.990386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:26.991797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.004990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/668] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.004990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.055289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e912eb-b7a2-42fb-a5ad-eaf46e891250 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.090735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.091383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.091663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.091947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.092805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:27.093163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.094596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.106554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/669] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.111509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.115394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25571874-0856-4e6b-a6dc-e68567746b96 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.123509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.144618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.154736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.155554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.155841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.156116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.169221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.170694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:27.188771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:27.189670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:27.190584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:27.190584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:27.190584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.190584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:27.190584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/670] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:27.193288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.230626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ebb0-3299-4ca0-9418-26177ee9a705 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.266062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.266862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.267152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.267422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.278071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.279619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.297574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da395efb-4759-4674-9fad-5dddde6fb996 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:27.297888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:27.298761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:27.299768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:27.299768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:27.299768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.299768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:27.299768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/671] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:27.300485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.338095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.338903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.339186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.339461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.350837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.352335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.364738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:27.369387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:27.370357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:27.371242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:27.371242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:27.371242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:27.371242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:27.371242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/672] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:27.371522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.405668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.411690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8f90-4fd4-40ae-a46f-6da16c8c7967 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.447059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.447808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.448155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.448526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.449439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=9bdf485ec3ca4f3785bb47f3706c4466) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:27.449755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.451232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.457835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/673] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/role_assignments?user.id=9bdf485ec3ca4f3785bb47f3706c4466&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.458664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.494774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe46aa-2084-459b-b363-85147db0012a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.535245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.536185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.536663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.537077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.538108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:27.538422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.539896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.559449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/674] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.559449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.612899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fa6b-176e-4b67-bd6a-29f92f52f457 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.650796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.662589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.663512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.663940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.664404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.665171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:27.665487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.666161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.666874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.680987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/675] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.681820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024dbaf-d031-4032-9764-d50fe7a1b06f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.682352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.733167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.733847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.734130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.734401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.735278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:27.736460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.737971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.752476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/676] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:27.753723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.816719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef833f9-ee93-4d46-a3fd-0be1a55c0b87 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.853297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.854099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.854388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.854668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.857290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46e74-3b7a-41c7-9f8b-65f5cdcde7e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:27.861963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.863840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:27.879001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:27.880170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:27.881188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:27.881668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.900074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:27.900949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.901235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.901531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.908330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:27.909889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-198c8b21-c463-45e7-a315-30178b460c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:27.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:27.925487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:27.926409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/678] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:27.926915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100018e8-8e5e-4a6c-a073-7b094c41c533 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:27.926915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100018e8-8e5e-4a6c-a073-7b094c41c533 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:27.927446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100018e8-8e5e-4a6c-a073-7b094c41c533 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:27.933019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/679] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:27.933019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100018e8-8e5e-4a6c-a073-7b094c41c533 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:27.936840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.944363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:27.995210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218dee9-4134-40de-90cb-7b76a5e55de8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.032460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.033545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.033929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.034649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.041199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100018e8-8e5e-4a6c-a073-7b094c41c533 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.044923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.047278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:28.063325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:28.064495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:28.065809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/680] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:28.066879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.079689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.080550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.081025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.081690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.082405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.082827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.084376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf010af-0645-4f55-8deb-f993baeae755 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.102151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/681] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.102151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.102151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.102151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.116352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:28.117645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:28.191452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.194695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5206ca-5b4e-4673-8389-d59eb260b858 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.202238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.208691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/682] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:28.209495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.232527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.233196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.233482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.233754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.234533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.234837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.236274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.252456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/683] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.252456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.326147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b06d7-2a2b-4ec5-800d-a20825787a26 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.356759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd96b-4ad2-492a-b7c0-732b4a74367c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.362601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.363422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.363777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.364148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.365538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.365954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.367864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.391188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/684] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.391188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.398174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.398836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.399118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.399391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.400222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=18bc17792b4d4879a1fd533206245694) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.400571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.402012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.408074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/685] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/role_assignments?user.id=18bc17792b4d4879a1fd533206245694&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.408716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.457177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.481727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.542008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7665c1-5126-4f93-b403-345cf5d63e6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.544038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77bb16-0fd2-4646-8a92-b3cd850bf0e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.585355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.586413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.586817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.587222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.591945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.591945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.591945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.592234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.592943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=357dd2446e9248cab4e559dc5ce931b1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.593883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.594932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.602002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:28.602332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/686] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/role_assignments?user.id=357dd2446e9248cab4e559dc5ce931b1&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.603162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:28.609634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:28.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:28.611756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:28.620523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/687] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:28.624639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.736464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.747202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1501a-50d4-414b-8bd2-13760b60f2b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.749815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbc52e-dd9f-497d-a7b3-56bd0aa749b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.774857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:28.791594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.791840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.792260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.792533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.792640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.792821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.792920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.793098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.793865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:28.794174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.795679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.806462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/688] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:28.813129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:28.886890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.889731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.933930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dbab9-9db0-473d-87a0-ba79bb6a7bab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:28.968248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:28.969085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:28.969368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:28.969643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:28.982154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:28.983625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:28.991624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.000583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/689] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:29.005399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:29.006483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:29.007589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:29.007589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:29.007589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:29.007589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:29.008065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.009538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/690] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:29.018644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.051688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.122037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47201d82-84d7-4e37-a8de-8b76b6bb6215 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.127463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9367cc-5797-4eec-af45-11bae97db428 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.161088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.162209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.162564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.162912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.164810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.166357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.166780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.167205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.168079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:29.168564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.170108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.174780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.176396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.188601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/691] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:29.190042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:29.195523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:29.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:29.197773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:29.197773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:29.197773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:29.197773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:29.203168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/692] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:29.209596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.297783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46416cd8-85f5-43ec-9dc4-612f2f2f0933 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.310834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.313838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.335159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.336277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.336757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.337224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.345528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.347425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:29.362636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:29.363645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:29.364593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:29.371711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/693] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:29.373264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.470394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfebc50-af6a-402d-8bab-19b175cd0306 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.479079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90131967-7dde-4569-b49c-af5ff1a2beb9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.514440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.515469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.515979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.516504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.517538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:29.518037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.520008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.542985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/694] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:29.544546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.554303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.554886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.555178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.555462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.556202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:29.556541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.557686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.558920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.589113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/695] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:29.601466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.654197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d95eb-fd3b-4984-a52f-89de61fe6d86 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.690404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.691438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.691941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.692440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.762660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.764473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101177a4-d5a4-49a6-9092-dc34fa8332b3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.777147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.822593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d97c4-3166-4d57-95c5-cc6cfe17fc62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:29.838132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:29.866016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:29.866797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.867082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.867361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.874474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:29.879730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:29.880043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/696] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 02:56:29.881861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9da6e-0534-4978-915a-139960e1a417 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.882294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9da6e-0534-4978-915a-139960e1a417 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.882801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9da6e-0534-4978-915a-139960e1a417 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.885034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/697] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:29.886107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9da6e-0534-4978-915a-139960e1a417 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1713fe85-643a-41f9-a336-295dcc8b59ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:29.895640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:29.896794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:29.897891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:29.904139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/698] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:29.904139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:29.904139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:29.904139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:29.919916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:29.921320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:29.990391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/699] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:29.991351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.003212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9da6e-0534-4978-915a-139960e1a417 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.033715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.048440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.048440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.048440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.048440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.060505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.063074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:30.085485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:30.086462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:30.087331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:30.087331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:30.087331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:30.087331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:30.087560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.089025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/700] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:30.090524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.092431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec40e6e-3105-4f21-9df1-6ad7df17dd76 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.127899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.128617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.128904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.129184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.129976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:30.130288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.131728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.143977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/701] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:30.144857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.225044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96db2c-d1e6-4c81-84cc-ea97b7900537 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.259090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62a465-194b-4d60-9d8a-58c5cbfd0368 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.265994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.267556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.268044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.268729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.283676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.286286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.295699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.296382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.296709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.296992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.297720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:30.298048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.299411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:30.310655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:30.311596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:30.312716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:30.312716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:30.312716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:30.312716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:30.312716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/702] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:30.316462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.317815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/703] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:30.318759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.329881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.338502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.435527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7815d-623a-45b9-a5f8-2d79af4d2a19 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.443436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aaa4e-595c-4a50-a966-1695f96c887c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.470306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.471296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.471296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.471657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.472598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=5b4f45c15c2949f5bd0e606b3cf00493) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:30.472915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.474385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.480835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/704] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/role_assignments?user.id=5b4f45c15c2949f5bd0e606b3cf00493&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:30.481938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.488377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.489607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.490097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.490616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.491668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:30.492276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.493918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.508521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/705] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:30.513556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.570840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.595405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a75dd-2636-4d76-8d95-5b56ffbc1580 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.607997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.632713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.633536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.633813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.634088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.638657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073586-912c-4855-a046-7e03c11da68e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.642328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.643035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:30.657425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:30.658432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:30.659669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/706] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:30.664972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.682981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.684030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.684648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.685125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.686158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.688015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.712435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/707] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:56:30 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:30.783743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd050a-6245-4eba-a764-476b46427c04 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.823831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:30.824461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.824750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.825026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.825917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:30.826233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:30.827782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70409a57-5970-45f1-a2a5-55e846c80450 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:30.831003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:30.843545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/708] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:30.849857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:30.852507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3530c-6359-4ea7-ac1b-d89d299b2a65 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.852943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3530c-6359-4ea7-ac1b-d89d299b2a65 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.853049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3530c-6359-4ea7-ac1b-d89d299b2a65 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.854637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/709] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:30.860102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:30.861052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:30.861052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:30.880833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:30.882065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:30.961659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/710] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 02:56:30.976250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d70fc-77a6-47d5-af5e-156f5c3c5aa4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:30.985293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:31.012117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:31.012822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.013110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.013396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.014244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:31.014562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:31.015024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:31.019255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:31.029595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/711] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:31.115456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed95ce-b1bd-49fc-9a4b-e98d465c1ad6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:31.156191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:31.171433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:31.172363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.172680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.173007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.173747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:31.175414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:31.197286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:31.198365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:31.199235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/712] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jun 21 02:56:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:31.204686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:31.330564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb998b-8915-4841-a13f-0033bbf9ecae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:31.377749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:31.378851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.379348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.379818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.380790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:31.381294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:31.383025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f18173-1fa8-40f9-adc8-bc25692aeb20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:31.392247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/713] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:31.398167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:31.536496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802fc73-9f62-401d-ad98-4bc20403f7de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:31.582571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:31.583343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.583535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.583813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.584567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:31.584881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:31.586314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89818600-7374-467f-a580-5debf59d6f75 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:31.597597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/714] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:31.697828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c2e6-6cdc-4b18-bee1-395d796ef6a7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.698482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c2e6-6cdc-4b18-bee1-395d796ef6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.698965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c2e6-6cdc-4b18-bee1-395d796ef6a7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.701090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/715] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:31.706839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:31.707213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:31.707495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:31.729205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:31.730593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:31.795550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:31.819156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/716] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:31.844813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78c2e6-6cdc-4b18-bee1-395d796ef6a7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:31.925290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:31.971690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78c2e6-6cdc-4b18-bee1-395d796ef6a7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:32.008022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:32.008889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:32.009215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:32.009500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:32.010167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:32.011598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:32.025994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:32.026987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:32.027813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/717] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:31 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:32.033101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:32.155977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e27708-6420-4396-808b-469ef0b79269 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:32.214683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:32.215380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:32.215668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:32.215942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:32.216696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:32.217007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:32.218409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f146b-030f-4cd8-996b-6799acbed36b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:32.228287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/718] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:32.233738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:32.384172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a907f-ef47-4809-b399-3d8be5cd231f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:32.416444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:32.428614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:32.429309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:32.429590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:32.429863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:32.430589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:32.430898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:32.432290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b2056-3cc2-4014-b061-99dfbd718cca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:32.441727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/719] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:33.298580 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35df59-16b7-4946-afcd-63358468d827 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.299115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35df59-16b7-4946-afcd-63358468d827 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.299115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35df59-16b7-4946-afcd-63358468d827 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.300714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/720] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:33.306253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.306717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.306805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.334552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:33.335767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:33.374053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fee2d-1e2f-4538-b6bd-536ce6dae138 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.374328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fee2d-1e2f-4538-b6bd-536ce6dae138 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.374668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fee2d-1e2f-4538-b6bd-536ce6dae138 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.376754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/721] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:33.388439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129996c-4b9e-45f6-be17-539aef10bc1c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.388439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129996c-4b9e-45f6-be17-539aef10bc1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.388439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129996c-4b9e-45f6-be17-539aef10bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.440678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f129996c-4b9e-45f6-be17-539aef10bc1c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:33.441810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f129996c-4b9e-45f6-be17-539aef10bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:33.472079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/722] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:33.504542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:33.511086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:33.555998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/723] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:33.557476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c62e-50a8-4fbf-9dc7-0adf81006a20 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.557768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c62e-50a8-4fbf-9dc7-0adf81006a20 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.558051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268c62e-50a8-4fbf-9dc7-0adf81006a20 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.559537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/724] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:33.579046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.579340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.579619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.604736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:33.606702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:33.639498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfac28c-3e86-411f-822e-73e8338db2c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:33.661578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:33.681705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:33.682644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.682941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.683259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.684998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/725] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:33.686119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:33.707445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:33.715581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:33.716743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:33.717790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:33.720146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/726] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:33.721195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:33.808082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b5790-9821-4c89-ade5-e9cfdcdc6f1e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:33.819238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:33.827953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5015b-8f9a-4d74-8207-00a29e9d59e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:33.847552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:33.848470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.849082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.849282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.860577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:33.865370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ec467-2399-42e5-b71f-755400318379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:33.866347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec467-2399-42e5-b71f-755400318379 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:33.866776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec467-2399-42e5-b71f-755400318379 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:33.867194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ec467-2399-42e5-b71f-755400318379 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:33.868860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:33.869995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:33.871031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:33.877616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/727] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:33.877616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:33.888105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ec467-2399-42e5-b71f-755400318379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-032ec467-2399-42e5-b71f-755400318379 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:33.898410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:33.899594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:33.900470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:33.904485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/728] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:33.905536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ec467-2399-42e5-b71f-755400318379 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:33.956140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.016015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f5f01-882d-40d4-a738-3a8891eadb6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.032326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ec467-2399-42e5-b71f-755400318379 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.063056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.063979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.064258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.064695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.066266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.066676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.068075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.070160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.071007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.071460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.071878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.072816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.073292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.074812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.078037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/729] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.079055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.089398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/730] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.090550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.118610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.191763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f6899-e5ad-44ba-bd8b-dd3062f83033 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.195311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.198932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c136b60-a2bb-48d7-bc92-bcc4c15a6753 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.226952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.227636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.227924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.228399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.228973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.229279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.230683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.236785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.237814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.238243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.238659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.244335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/731] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.245268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.251321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.253211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:34.271569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:34.272675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:34.273675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:34.273675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:34.273675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.273675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:34.275528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/732] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:34.276475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.339957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.349339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4250af-f1c7-47c3-8cdd-aea2ef236b88 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.383418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a1c4-7d55-4920-8c7c-8e07d3cc9e35 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.385569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.386456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.386803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.387842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.398504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.400158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:34.418128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:34.419240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:34.420350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:34.420350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:34.420350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.420350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:34.425944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/733] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:34.425944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.429473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.431355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.431721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.432069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.447467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.449674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:34.471017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:34.472201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:34.473352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:34.473352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:34.473352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:34.473352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:34.474360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/734] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:34.475217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.481393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.536247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898339-723f-413a-ba64-e66df38496d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.573599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.574269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.574546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.574826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.575557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.575860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.577294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb952c-4f13-4133-adef-899429845269 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.589646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/735] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.591001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6c6f-e88f-4f69-b30f-be4ec50fccc4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.591288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6c6f-e88f-4f69-b30f-be4ec50fccc4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.591564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d6c6f-e88f-4f69-b30f-be4ec50fccc4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.592961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/736] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:34.593549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d6c6f-e88f-4f69-b30f-be4ec50fccc4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.600002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.602343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bf59a-c6af-4049-a7d6-76d9012f7d6e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.640322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.641141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.641461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.641741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.642471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.642784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.644394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.661993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/737] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.662907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.698994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d6c6f-e88f-4f69-b30f-be4ec50fccc4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.728654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.734894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.735568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.735849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.736121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.736875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:34.737375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.738921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21035-0706-4782-824c-b844a8679366 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.750584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/738] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:34.751987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.752271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.752648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.769682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:34.770866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:34.776546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6985c9-50eb-4f78-96b0-1b668b13e098 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.812926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.813668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.813953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.814227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.822117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:34.823580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:34.832047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:34.839902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:34.840900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:34.842319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:34.845789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/739] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:34.846809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.851333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/740] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:34.852254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:34.952306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25cd3d-81c2-4d09-a3ff-585f391befd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.961869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac97da6-3188-41d7-ba33-7f2e44cb1ade None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:34.978534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:34.989933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.990741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.991055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.991332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.996266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:34.997112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:34.997393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:34.997652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:34.998602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.000048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.005290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.006721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:35.015224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:35.016122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:35.017112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/741] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:35.017702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:35.021756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:35.027294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/742] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:35.028498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.078030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.122170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764c7-d831-4f37-a68a-2f6d55bab379 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.124733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6771d-efe2-467a-be0d-92855c23a2a6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.158582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.159345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.159714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.160044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.160776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.162073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.162196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5af1e-6193-470c-be2e-867d44fdcd92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.162760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.163053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.163340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.164278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:35.164627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.166392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.177844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/743] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:56:35 2022] POST /identity/v3/services => generated 251 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:35.179236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f09e0-3ff4-46db-b687-ff2fef0c3d11 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.179535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f09e0-3ff4-46db-b687-ff2fef0c3d11 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.179814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f09e0-3ff4-46db-b687-ff2fef0c3d11 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.181316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/744] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:35.181945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f09e0-3ff4-46db-b687-ff2fef0c3d11 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.197192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/745] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:35.198909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.255067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.316807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f09e0-3ff4-46db-b687-ff2fef0c3d11 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.326340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312dff8-0962-4ddb-a304-f98b6ae7277e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.330040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.363110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.363833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.364093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.364376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.365404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:35.365721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.367750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31b8a-bd35-4e6b-9363-5093e57d5df9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.367860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.368154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.368487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.368771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.369792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:35.371089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.372404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.386771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/746] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:35.387899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.388192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.392448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.395054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/747] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:35.396593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.407602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:35.408692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:35.480469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/748] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 02:56:35.481124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.502485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.506533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ada65-214b-4bbb-bc58-2e12b5b64f6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.546574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.547227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.547513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.547791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.597770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.601581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0970db-291e-465f-a27d-181aba752fc5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.623633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.625152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.639730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.640482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.640774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.641112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.714331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.716181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.726572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.735021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/749] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:35.735998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.815357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.820784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/750] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 02:56:35.821772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.847180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4c67d-5ac0-475d-a124-b15ef67a5a0e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.882435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.883311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.883597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.883879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.894297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.896163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:35.915655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:35.916693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:35.917591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:35.917591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:35.917591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:35.917591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:35.917591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/751] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:35.917877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:35.933431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eff2f-64f9-4c03-af31-f70fdbc65d58 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:35.974567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:35.975236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:35.975520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:35.975792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:35.976545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:35.976996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:35.978083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:35.993426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/752] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:56:35 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:35.994277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.040438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77cbd-fe3f-4df5-9f98-51621bb9acf2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.062828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.097875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.099078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.099558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.100038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.110341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423689c-dab7-410d-aff8-cc2bd6e2a7b5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.113607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.115412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:36.136691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:36.137637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:36.138535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:36.138535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:36.138535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:36.138535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:36.140675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/753] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:36.141714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.153510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.154336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.154617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.154889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.167876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.169433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:36.187348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:36.188494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:36.189401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:36.189401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:36.189401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:36.189401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:36.189401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/754] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:36.189980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.265349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.268657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733245b9-008a-47d4-ab19-658ae2da5eee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.308843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.309609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.309960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.310298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.311095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:36.311406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.312869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.320364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346ce76-8e75-4f41-a806-23e7d6319109 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.331429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/755] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:36.331429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.337633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.371635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.373254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.373746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.374211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.375135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:36.376592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.378745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.397780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/756] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:36.397780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.448647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43372a-f7fe-42be-82c8-246be461cdee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.489745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.490586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.490966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.491396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.492221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:36.492778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.494245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.507796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/757] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:36.508813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.523254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23a95-ab40-455d-8ffb-7d97e7e99afc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.534130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.571761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.573039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.573418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.573788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.582248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.584320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:36.602020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:36.602935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:36.603822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/758] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:36.604304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.635612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a331f26-5206-4c9e-be16-28a8a482b5ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.642135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.677189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.678517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.679311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.680061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.688920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.691644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:36.709245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:36.710193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.711103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:36.714020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/759] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:36.715235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.725670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3fb0f-69d3-41b6-b286-074425be0da7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.766836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.768462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.768896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.769264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.778451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.780087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:36.797031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:36.798145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:36.799040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:36.801605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/760] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:36.802717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.813496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.836224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1512-dd9e-4741-8f0a-99665cd204b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.873621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.874483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.874832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.876247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.876423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:36.876821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.878491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.891832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/761] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:36.892862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:36.898624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:36.909197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695186-78ff-47fc-a3d4-65498b81e41b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:36.944601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:36.945552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:36.946031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:36.946510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:36.947505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:36.948021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:36.949714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:36.962834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/762] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:36.964021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:37.005091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e75d2-dde8-456c-b27a-370fdf2207a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.043196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.043917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.044205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.044504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.045392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:37.045704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.047149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.057617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:37.062239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/763] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:37.063894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:37.069699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa78d0-770a-415d-a982-5fe6e4e1fe0b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.105308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.106597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.107077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.107546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.109809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=1c16671fd5844a77ac6e56dff36f86f8, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.111497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.130044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/764] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:36 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:37.131419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:37.146010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:37.202079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb1f60-0e91-4066-8977-4bde36da3164 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.249447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad57c54-cc73-42bd-a4dd-427b9f406bb9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.259070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.259743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.260028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.260306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.269636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=41a3f61f1038445aa7ab1224445fb2e3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:37.269966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.271497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904698ca-63e1-4363-b938-f0cc56ee0450 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.278382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/765] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:37 2022] GET /identity/v3/role_assignments?user.id=41a3f61f1038445aa7ab1224445fb2e3&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:37.290840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.292055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.292625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.293129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.294239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=5b4f45c15c2949f5bd0e606b3cf00493) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:37.294782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.296599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45502bb8-edf7-43f6-860e-a8a4b1aa4efa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.303990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/766] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:37 2022] GET /identity/v3/role_assignments?user.id=5b4f45c15c2949f5bd0e606b3cf00493&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:37.418867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd54b-a399-45b5-8a43-108b8905ea23 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.419215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd54b-a399-45b5-8a43-108b8905ea23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.419435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fd54b-a399-45b5-8a43-108b8905ea23 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.421216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/767] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:37.428180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.429197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.429670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.449987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:37.451634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:37.528848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/768] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:37.554779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fd54b-a399-45b5-8a43-108b8905ea23 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:37.626283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:37.677851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fd54b-a399-45b5-8a43-108b8905ea23 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.713025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.713850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.714131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.714404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.715066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.716458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:37.728961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:37.729939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:37.731204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/769] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jun 21 02:56:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:37.736362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:37.740017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:37.885223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66dc11-6433-40d4-b388-b18ecc9f46be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:37.935844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:37.936550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:37.936834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:37.937104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:37.937825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:37.938135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:37.939539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:37.948437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/770] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:37.954212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.076029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5eb66-6a5b-4bda-a6d5-a8f884451851 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:38.106782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:38.120715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:38.121397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.121683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.121958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.122672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:38.122980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:38.124365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a91e3-8545-402f-b6d0-18e6d9941e60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:38.130010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/771] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:38.136028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.216101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:38.256062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542bc23-fc84-4e13-b646-f0e3ec57f79d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:38.285874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b81-65a1-4091-b526-9f9bfafb2976 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.286075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b81-65a1-4091-b526-9f9bfafb2976 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.286365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b81-65a1-4091-b526-9f9bfafb2976 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.288073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/772] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:38.293063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.293397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.293718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.295364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:38.296045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.296370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.296684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.297493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] RBAC: Adding query filter params (interface=public, service_id=143f6c921af74a12bef2d15946478339, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:38.297813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:38.299215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:38.308103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/773] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:56:38 2022] GET /identity/v3/endpoints?service_id=143f6c921af74a12bef2d15946478339&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:38.317367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.325968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:38.326656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:38.430437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/774] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:38.464518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.482896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d6d85-b25b-4af5-8a1f-e75212f5980f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:38.541533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:38.550320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:38.551189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.555231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.555519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.556233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:38.557666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:38.575335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/775] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:56:38 2022] GET /identity/v3/services => generated 772 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:38.617139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a0b0d-0ec0-4a41-8661-8149c4c62857 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:38.651394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:38.672445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:38.673338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.673734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.674024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.674844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:38.676488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:38.689827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:38.690899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:38.692674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/776] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:38 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:38.697620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.820622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f746-a1da-4f50-b870-e716a7e47430 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:38.863643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:38.864340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:38.864649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:38.864979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:38.865712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:38.866075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:38.867541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:38.875911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/777] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:38.881646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:38.989706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afdd84-0511-4fb3-be05-15d5c948a69a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:39.025610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:39.026286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.026613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.026899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.027613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:39.027923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:39.029325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:39.040636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/778] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:39.046451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:39.129078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:39.196673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fdffa-eab4-4a3f-9df0-cb4f435bef60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:39.237487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:39.238320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.238609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.238887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.239700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] RBAC: Adding query filter params (interface=public, service_id=8d2b8fe0f42546b792b3278d8c10b78f, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:39.240071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:39.241465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61eb37-d412-4fd4-a64c-ff28e63d6b6f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:39.250585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/779] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:56:39 2022] GET /identity/v3/endpoints?service_id=8d2b8fe0f42546b792b3278d8c10b78f&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:39.256369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:39.319215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:39.372567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c812-307e-4151-a297-52e40585e1eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:39.421934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:39.422581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.422865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.423142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.423845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:39.425309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28b46-2899-47e9-92d7-3db1f2b1dd67 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:39.437278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/780] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:56:39 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:39.712628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d0d2-9626-4c51-94f7-477e6f6185c3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d0d2-9626-4c51-94f7-477e6f6185c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.713143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1d0d2-9626-4c51-94f7-477e6f6185c3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.714677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/781] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:39.720258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645efa2-0c31-469b-a560-eef9bbd28c10 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.720802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645efa2-0c31-469b-a560-eef9bbd28c10 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.720964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645efa2-0c31-469b-a560-eef9bbd28c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.743096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0645efa2-0c31-469b-a560-eef9bbd28c10 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:39.744289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0645efa2-0c31-469b-a560-eef9bbd28c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:39.817443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/782] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:39.840057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d0d2-9626-4c51-94f7-477e6f6185c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:39.853399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295eaad-8219-4d76-97e3-3fc91a8aed2a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.853741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295eaad-8219-4d76-97e3-3fc91a8aed2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.853960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295eaad-8219-4d76-97e3-3fc91a8aed2a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.855531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/783] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:39.861646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d13356-b4b5-4b5b-9965-bed6510d7df4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:39.862299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d13356-b4b5-4b5b-9965-bed6510d7df4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:39.862772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d13356-b4b5-4b5b-9965-bed6510d7df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:39.905527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-74d13356-b4b5-4b5b-9965-bed6510d7df4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:39.907100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74d13356-b4b5-4b5b-9965-bed6510d7df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:39.963438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1d0d2-9626-4c51-94f7-477e6f6185c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:39.970214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:39.999773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.002477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/784] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:40.006572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88109932-8afd-4be9-b830-4e4add7fcd3a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.007354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88109932-8afd-4be9-b830-4e4add7fcd3a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.007858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88109932-8afd-4be9-b830-4e4add7fcd3a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.014371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/785] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:40.015479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.016304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.016610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.016888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.018682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.018986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.019322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.033666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:40.037070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:40.038330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:40.043833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:40.044994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:40.046090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:40.049754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/786] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:40.051384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.115366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/787] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:40.116245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.187153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7ed8-d11f-4be7-b15b-7f87134e120f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.232181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98945a5-e846-4691-8a8f-669240872a22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.232820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.233638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.233921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.234194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.236558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.252255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:40.260263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:40.261237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:40.262062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/788] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:40.262762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.281944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.283371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.283933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.284428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.285404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:40.286408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.288584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.301191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/789] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:40.302274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.316003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.385630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73193b83-5759-411c-9c5e-368e48e71b8d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.418934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055043ce-28cc-4eae-8314-e3571b97364c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.422528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16303-9678-45b6-9702-085270beae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.423344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16303-9678-45b6-9702-085270beae60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.423627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16303-9678-45b6-9702-085270beae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.423903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b16303-9678-45b6-9702-085270beae60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.434881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b16303-9678-45b6-9702-085270beae60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-74b16303-9678-45b6-9702-085270beae60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:40.445732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:40.446737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:40.447568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/790] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:40.448261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16303-9678-45b6-9702-085270beae60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.468527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.470020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.470502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.471022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.472208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:40.473316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.475464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.488122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/791] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:40.489255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.524953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.567417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b16303-9678-45b6-9702-085270beae60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.570600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.604352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.605205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.605533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.605934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.617591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.619120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236921b-eef2-437a-94d0-caee37ee0c61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.619233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:40.637458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:40.638405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:40.639281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:40.639281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:40.639281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:40.639281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:40.639281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/792] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:40.640109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.661813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.662542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.662831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.663112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.663968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:40.664282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.665809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.675688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/793] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:40.676682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.770539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e347b9-a517-45f0-8b2e-79a716f0877c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.777033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.793460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b1372-4021-4fa8-8052-0806791cbfe5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.811793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.812645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.813135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.813241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.825414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.826930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.837062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:40.838020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:40.838495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:40.838987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:40.839900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:40.840399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:40.841999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:40.844681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:40.845554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:40.846537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:40.846537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:40.846537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:40.846537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:40.846537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/794] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:40.847587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.858650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/795] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:40.860190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:40.864061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:40.959131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc964a27-1251-40cf-931e-b779e3072437 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:40.995125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e8a8-11d6-40fc-8965-ff218c54f9ff None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.003390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.004519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.005008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.005576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.017582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.019523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.031176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.031783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.032069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.032404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.033169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:41.033534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.035419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:41.039653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.040553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.041421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:41.041421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:41.041421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:41.041421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.043679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/796] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.044776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.051855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/797] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:41.051855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.062811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.116733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.160208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964467-6ed6-4856-bd65-7449eac6fbd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.161272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31b4b8-7628-4e47-82e8-09fd6ed7f4e1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.195969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.196656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.196953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.197211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.197929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:41.198235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.199608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.202918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.203942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.204401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.205029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.212280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/798] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:41.213191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.215006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.216688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a6e1ecde-5ca2-447a-b4c8-865d8061a467 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:41.231515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.232546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.233486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/799] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.234741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3784-2a2e-4043-9ebe-8e9df58b7b8d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.234741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3784-2a2e-4043-9ebe-8e9df58b7b8d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.234741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c3784-2a2e-4043-9ebe-8e9df58b7b8d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.240600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/800] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:41.240600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c3784-2a2e-4043-9ebe-8e9df58b7b8d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.300228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.333621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51cfef-b549-4494-936e-a9e5ddd1f1c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.345305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c3784-2a2e-4043-9ebe-8e9df58b7b8d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.368057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.368919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.369141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.369256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.369469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.378030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.379505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.384982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.385843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.386155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.386429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:41.394888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.395828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:41.401676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:41.402212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.402293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/801] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.402363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.402430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bd2ac15c-b480-4379-bf28-64128914ec33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:41.419345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.420308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/802] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.422562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.424160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.440506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:41.441700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:41.508226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73ac76-b771-452a-988b-108a10ef2060 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.510891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/803] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:41.511767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.549228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.557034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.557628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.557916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.558191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.559028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:41.559340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.560801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.579147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/804] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:41.580718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.627315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3f1ca-0165-43f1-8e29-295dee15311d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.638874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.664244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.664957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.665240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.665513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.666362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:41.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.668138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.686702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/805] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:41.687354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.695352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a8a2-96e6-49ab-8b36-9e5a325401b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.735389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.736123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.736436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.736752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.738053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:41.738369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.739828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.762238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/806] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:41.762238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.798485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834b7bb-0a5a-4e63-8d19-402436b299a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.808442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.836806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.838035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.838317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.838589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.839600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.841337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.853675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:41.855464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/807] 10.222.0.16 () {56 vars in 1129 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.856302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.865553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a8053-4fd3-4a91-97c1-0925d817ae7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.886954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:41.899816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.900651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.900941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.901220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:41.911332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:41.912825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:41.930020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:41.930935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:41.931821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:41.931821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:41.931821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:41.931821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:41.931821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/808] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:41.932191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:41.957691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b566a-a38f-4b48-a1b3-50833709a6f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:41.992053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:41.992883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:41.993169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:41.993442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.004620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.006079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-efc46d65-8503-46d8-923f-f81c6f7bf6ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:42.023877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:42.025099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/809] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:42.026457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b67bfd-6533-41fa-8f61-5858c45a2dbf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.026851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b67bfd-6533-41fa-8f61-5858c45a2dbf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.026992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b67bfd-6533-41fa-8f61-5858c45a2dbf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.028366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/810] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:42.028947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b67bfd-6533-41fa-8f61-5858c45a2dbf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.035355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.039923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7387e-4a35-4bb0-a73a-4d7ce61b673b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.074890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.075473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.075760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.076061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.114750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.132444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b67bfd-6533-41fa-8f61-5858c45a2dbf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.147584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.149089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.166448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.167125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.167409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.167680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.168400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:42.168741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.170134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.174779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/811] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:42.175703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.249414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.255089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/812] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:56:42.256362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.284272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ff205-3510-4cdd-862c-41dba431baeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.318736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.319406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.319688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.319966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.320731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:42.321041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.322441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772efb0-789a-4dfc-870c-5a4121a05d8e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.334758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/813] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:42.335925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.336192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.336495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.352015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:42.353243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:42.355262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.359750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a747c-4c59-489f-8cf1-3145ee8f775c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.394892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.395570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.395860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.396138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.396896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:42.397210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.398601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.410412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/814] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:42.411201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.428631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/815] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:42.429470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.479242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.520873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d171269-fa3e-49b5-bd21-bde15401239e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.527883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48ed72-b996-4bb0-865a-1f51c85c1147 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.555457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.556142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.556449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.556723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.557441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:42.557751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.559164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.561757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.562556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.562833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.563107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.563994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/816] 10.222.0.16 () {56 vars in 1124 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:42.564800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.570948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.574795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.576646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:42.594141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:42.595080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/817] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:42.596013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.672069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6359b0-994b-4529-9dba-8a68e1a879f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.694552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.699732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3724f40-851f-4829-a582-1f1f577e6cba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.706258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.707036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.707300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.707572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.714230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.715691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:42.729836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:42.730726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:42.731598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/818] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:42.732080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.735761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.736614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.736934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.737757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.744603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.746103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:42.761061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:42.762058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:42.762946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/819] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:42.763442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.814448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:42.839777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23040-cabc-4994-b225-63a59b997fe4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.865966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5d58-4e5e-4102-8168-07cadf2928a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:42.873928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.874735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.875018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.875639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.876296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.877757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:42.889308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:42.890295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:42.891133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/820] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:42.891928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.904814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:42.905450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:42.905733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:42.906008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:42.906782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:42.908180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:42.929295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/821] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:56:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:42.930112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:42.944779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.007236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d95ee-6e15-47ba-ac9d-0025ccbe1995 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.053985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.054651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.054940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.055219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.055963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.056276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.057643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c2fd4-99e9-4a0c-b1dc-d24ea3423a49 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.057773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.059405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.075690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/822] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.077408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.105436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.106103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.106388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.106664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.107477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.107790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.109276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.122761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/823] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.123883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.203419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363ef0-816e-4d32-8e05-73cf630111e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.205337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.244872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.245664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.245952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.246234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.247039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.247354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.248883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.263207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/824] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.264056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.349941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741d92-0b5a-4479-8f78-a55d54d4ab93 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.374332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50df04b-40e6-42a5-8d5d-4e55beb45509 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.386864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.387547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.387833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.388117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.388900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.389216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.390659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.396451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/825] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.396807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.415145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.415972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.416264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.416568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.423620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.425342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.427341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:43.441648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:43.442576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:43.443489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/826] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:43.443970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.469103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.512794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd52cc-d549-4aad-a62c-0bf718bab3e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.548897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d7816-5f2d-4b30-90b6-23a7f5d9d269 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.549373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.550353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.550631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.550901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/5b4f45c15c2949f5bd0e606b3cf00493/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.552916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=5b4f45c15c2949f5bd0e606b3cf00493, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.554427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.569397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/827] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:43 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/5b4f45c15c2949f5bd0e606b3cf00493/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:43.570001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.597169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.598551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.599057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.599531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/41a3f61f1038445aa7ab1224445fb2e3/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.619451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=41a3f61f1038445aa7ab1224445fb2e3, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.621377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.641566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/828] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:43 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/41a3f61f1038445aa7ab1224445fb2e3/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:56:43.642938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.732969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.735033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:43.775990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f39086-0be8-4988-8ab1-c0cc9e9239a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.782513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0b8f-649b-48eb-9958-53988ca6487e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.824307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.825029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.825319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.825602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.826522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.826842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.828467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.834336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:43.835722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:43.836221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:43.836746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:43.838403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:43.838938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:43.841539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:43.843802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/829] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.848667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/830] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:43.853682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.855338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:43.984178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a54d12-3acb-4b33-b9ef-8a5abf8d1a20 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:43.996607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964bc7d-cbc5-427d-96f7-446552f95a2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:44.020626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:44.021660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:44.022206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:44.022689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:44.023652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:44.025287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d510ca-2ec2-43b3-9161-0d810156ce48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:44.051780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:44.052744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/831] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:56:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 02:56:44.066325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:44.067398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:44.067900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:44.068366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:44.069441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:44.070052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:44.071762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada52f-1d8f-4201-a0fe-c27fb63caffb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:44.079326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/832] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:43 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:45.066215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b11c5-1206-402b-8cb7-d78f030c8814 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.066755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b11c5-1206-402b-8cb7-d78f030c8814 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.066755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b11c5-1206-402b-8cb7-d78f030c8814 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.068372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/833] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:45.073725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.074111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.074289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.091393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:45.092620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:45.167537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/834] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:45.189377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b11c5-1206-402b-8cb7-d78f030c8814 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:45.198508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:45.294410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b11c5-1206-402b-8cb7-d78f030c8814 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:45.329065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:45.329794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.330076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.330350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.331098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:45.332503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:45.344891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:45.345989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:45.347312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/835] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:45.357651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:45.461804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcff753-3719-4c7e-9f58-45aae9e38418 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:45.485299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:45.498286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:45.498968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.499253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.499529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.500604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:45.501205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:45.502938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b224f-bf25-4ec9-ad85-1367a31d4319 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:45.508567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/836] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:45.514097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:45.585887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:45.619148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580e204-aeb1-4e1e-8364-1fe9bbd160b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:45.653251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:45.653928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.654208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.654478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.655193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:45.655498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:45.656914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30772d73-03bd-405c-8027-b540ae47d46b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:45.662281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/837] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:45.906991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce2e0-65e7-445f-9cda-efa5e765b919 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.907169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce2e0-65e7-445f-9cda-efa5e765b919 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.907451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce2e0-65e7-445f-9cda-efa5e765b919 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.909290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/838] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:45.914849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.915138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.915413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.932992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:45.934186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:45.942580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf55ca-8ef6-4de2-a331-f6fd827e0558 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.942868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf55ca-8ef6-4de2-a331-f6fd827e0558 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.943142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf55ca-8ef6-4de2-a331-f6fd827e0558 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.944778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/839] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:45.949693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:45.949979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:45.950257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:45.968852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:45.970118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:46.008539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/840] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:46.027730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.043160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/841] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:46.068214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.090190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.129303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c255d0e-ac70-417f-8f60-ace28496d1c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.174178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.175270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.175696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.176132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.180786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5216661-161d-4ca8-9b6a-a5ec2a4b45af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.187142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:46.195261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:46.196442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:56:46.197481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:46.199208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.204483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/842] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:46.206004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.219393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.220295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.220623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.220902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.232048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:46.240306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:46.241585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:56:46.242682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:46.246598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/843] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:46.247614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.324981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e325aba-a68d-4b2f-90ac-ec3cbc95f64b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.349314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.358334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516682f-66fc-4630-9487-82f520a51481 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.361325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.361594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.361931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.362206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.363047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:46.363354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.364759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47f3f0-bac4-4be5-bbef-b883ecd7e4fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.378405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/844] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:46.378405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d83-8a7a-4dcd-b37f-ef948d8e9e3e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.378405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d83-8a7a-4dcd-b37f-ef948d8e9e3e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.378405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d83-8a7a-4dcd-b37f-ef948d8e9e3e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.378405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/845] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:46.381616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352d83-8a7a-4dcd-b37f-ef948d8e9e3e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.402336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.403021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.403307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.403703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.404494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:46.404895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.406336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fa127-c322-4784-8d03-e887df6ecc2c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.419759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/846] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:46.419759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.419759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.419759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.436987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:46.438186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:46.472006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.492008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352d83-8a7a-4dcd-b37f-ef948d8e9e3e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.520906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/847] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:46.520906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.537536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.538354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.538643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.538923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.550260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.551774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:46.569707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:46.570594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:46.571486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:46.571486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:46.571486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:46.571486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:46.575780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/848] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:46.576666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.618609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.624979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ee892-f638-40da-9bb6-e0af4ed551ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.661926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.662737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.663026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.663313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.674385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.675757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.682980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8368-3939-4d6b-8447-55ff91c5b542 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:46.693697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:46.694802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/849] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:46.695685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.722043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.723968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.724832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.725177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.725433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.726167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.727557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:46.740455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:46.741424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:46.742677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/850] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:46.743708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.814229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58304aa1-f653-4262-99f1-b1fffcb46d07 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.850489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.850903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5088a-f36a-4fb1-ac87-e42f8e46dade None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:46.851412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.851794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.852188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.852572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.853673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:46.854075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.855681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.873094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/851] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:46.873094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.893611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:46.894354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:46.894637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:46.894915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:46.895761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:46.896069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:46.897689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:46.914810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/852] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:46.914810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:46.959156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:46.978820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0626df-bd64-4517-a1a5-7a0495c3b65b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.016359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.017226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.017614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.018020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.018120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f59c-e337-401f-b02c-05942047e81b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.018895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:47.019304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.020828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.025796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/853] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:47.026640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.063382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.064274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.064656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.065055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.073329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.074925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:47.090679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:47.092084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/854] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:56:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:47.093051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.119310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.132037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cced7d-de95-47fd-85b7-40aa530d567d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.173168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.173997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.174318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.174586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.182401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.183850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:47.198586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:47.199556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/855] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:47.200567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.205816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.208350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba9c51-4a8c-4165-bc3c-fcab27f44895 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.247670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.248235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.248547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.248821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.249530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:47.249840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.251699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.268618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/856] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:47.268618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.343065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71958d4a-0151-4e56-abcf-182785011482 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.371134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.390126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.390456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937181-c7ef-4819-9c4c-2d54dfc9f219 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.390823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.391105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.391379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.392256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:47.392603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.395701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.421814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/857] 10.222.0.16 () {56 vars in 1135 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:47.422771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.432293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.433037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.433321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.433599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.434450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:47.434818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.436308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.459711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/858] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:47.460710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.498767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.547606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326177e8-bf86-4e5d-8864-d2b426ef8b9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.565950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9cc1c-a199-4173-a3c5-6300bae76424 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.595220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.596178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.596485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.596851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.602786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.603447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.603730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.604004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.621568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.682410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.683887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7267c7-cc7c-416d-9766-f6fdf0e1aff4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.688959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.690524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040b622-0a40-4eff-9a99-2d5421588905 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.764090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.796460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/859] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:56:47.796460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ef38-81b3-483c-bd2e-661e9d17af1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.796460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ef38-81b3-483c-bd2e-661e9d17af1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.796460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734ef38-81b3-483c-bd2e-661e9d17af1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.800250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/860] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:47.801195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734ef38-81b3-483c-bd2e-661e9d17af1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:47.855185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:47.915540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734ef38-81b3-483c-bd2e-661e9d17af1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:47.931807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/861] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:56:47.931807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.931807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.931807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.947579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:47.948891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:47.957187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:47.958156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:47.958527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:47.958902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:47.969933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:47.971881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:47.990392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:47.991507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:47.992633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:47.992633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:47.992633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:47.992633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:47.996295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/862] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:47.997203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.024248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/863] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:48.025842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.097936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.104063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658eb2b-4a9c-4005-8ebe-dc62ff2df6c3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.139059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.139410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.140557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.140894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.141120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.148143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206b467-ac9c-4b29-896b-61d2d283cfc0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.153815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.155769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:48.174948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:48.175845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:48.176775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:48.176775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:48.176775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:48.176775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:48.176775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/864] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:48.177062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.188544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.189620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.189984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.190258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.190992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.191661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.193394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.205299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/865] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:48.206000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.290545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dfcbf-cf0f-4a90-b4ef-b3b1485afcf8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.310085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd0c3-d70d-4e38-9d94-ffb24a716948 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.329426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.330610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.330895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.331179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.332056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.333596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.360446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/866] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:56:48 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:48.360446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.371683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.372501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.372798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.373083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.373987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.374533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.376805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.378280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.405386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/867] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:48.411582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.424708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.497839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee7dad-76d9-4b8d-ab3b-08bc23d8285b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.527867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943aa4a-9a2c-483a-ae27-f2216a7a2f91 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.536208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.536208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.536541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.536749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.544078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.545537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:48.560172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:48.561296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:48.562286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/868] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:48.569142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.571770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.572683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.573885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.574171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.582987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.584562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:48.599652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:48.600635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:48.601505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/869] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:48.606693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.659524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.682271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.700882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d271-07f1-425f-b90c-6f99f84cbc2b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.727632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4caef4-383d-45b8-a03a-714a4b48bae9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.746272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.747262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.747767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.748242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.749886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.750869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.753436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.763816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.764521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.764810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.765088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.765900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.766211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.767694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.776988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/870] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:48.779457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.783094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/871] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:48.788959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:48.896225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fe26-b957-48cd-b2aa-7ca410c17f17 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.912330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518aa5-9c22-4140-9bcf-abd4a028ab06 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:48.927392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.938655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.939583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.940071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.940511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.941789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=41a3f61f1038445aa7ab1224445fb2e3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.942185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:48.943358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.945062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c14f91-3a6a-4dd4-8ab2-a86c11c308ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.951500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:48.952251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:48.952644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:48.952993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:48.960453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=5b4f45c15c2949f5bd0e606b3cf00493) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:48.960453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:48.960453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ac2c1-3df4-4f24-aece-ede8cec882e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:48.961275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/872] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/role_assignments?user.id=41a3f61f1038445aa7ab1224445fb2e3&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:48.968537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/873] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:48 2022] GET /identity/v3/role_assignments?user.id=5b4f45c15c2949f5bd0e606b3cf00493&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:49.363330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf2e9-c9fe-400e-b63c-e1a793ce8ce5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:49.363863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf2e9-c9fe-400e-b63c-e1a793ce8ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:49.363863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedf2e9-c9fe-400e-b63c-e1a793ce8ce5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:49.365526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/874] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:49.370955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d45e-3ff8-4ee6-828a-15945ee6b9b9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:49.371395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d45e-3ff8-4ee6-828a-15945ee6b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:49.371572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d45e-3ff8-4ee6-828a-15945ee6b9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:49.389118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-02b7d45e-3ff8-4ee6-828a-15945ee6b9b9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:49.390323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02b7d45e-3ff8-4ee6-828a-15945ee6b9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:49.459459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/875] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:49.481851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedf2e9-c9fe-400e-b63c-e1a793ce8ce5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:49.591067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedf2e9-c9fe-400e-b63c-e1a793ce8ce5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:49.629075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:49.629762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:49.630047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:49.630325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:49.631072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:49.632508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45900d-dd54-43ad-8881-c49daefc7f49 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:49.650828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/876] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:56:49 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:50.552895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695dc9-a2c0-40b4-b569-fdf532eaa67d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:50.553442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695dc9-a2c0-40b4-b569-fdf532eaa67d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:50.553442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695dc9-a2c0-40b4-b569-fdf532eaa67d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:50.555133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/877] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:50.560574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:50.560972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:50.561138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:50.578561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:50.579771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:50.629754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:50.662685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/878] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:50.680182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10695dc9-a2c0-40b4-b569-fdf532eaa67d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:50.788265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10695dc9-a2c0-40b4-b569-fdf532eaa67d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:50.811932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:50.823931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:50.824778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:50.825277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:50.825726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:50.826447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:50.828042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:50.839915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:50.840946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:50.841784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/879] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:50.850002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:50.964204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387fe17-4568-4175-a8da-6fead09d886f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.000043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.001137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.001423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.001699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.002478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.002789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.004364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61ef16-5fc0-4733-843c-b612f40dee39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.013946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/880] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.017493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.099926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975e30-3513-42fb-af89-0ed2e8a6f85d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.100119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975e30-3513-42fb-af89-0ed2e8a6f85d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.100538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975e30-3513-42fb-af89-0ed2e8a6f85d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.102210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/881] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:51.107225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc31f-7fa1-4358-a73e-a1174e984386 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.107582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc31f-7fa1-4358-a73e-a1174e984386 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.107930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc31f-7fa1-4358-a73e-a1174e984386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.121603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76a1ef-6eff-42a5-aab2-3133c6fc30f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.125233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-79abc31f-7fa1-4358-a73e-a1174e984386 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:51.126771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79abc31f-7fa1-4358-a73e-a1174e984386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:51.168951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.172993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.173675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.173955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.174303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.175729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.176045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.177830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447b990-d500-4798-a21a-45f58d637322 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.183970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/882] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.185499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f80a-7778-4614-8a68-93c694f486fe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.185844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f80a-7778-4614-8a68-93c694f486fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.186121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f80a-7778-4614-8a68-93c694f486fe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.188454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/883] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:51.189750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836f80a-7778-4614-8a68-93c694f486fe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.216750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/884] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:51.216750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009361-23c3-4a78-872a-463262a10421 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.216750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009361-23c3-4a78-872a-463262a10421 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.216750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6009361-23c3-4a78-872a-463262a10421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.243307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a6009361-23c3-4a78-872a-463262a10421 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:51.244851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6009361-23c3-4a78-872a-463262a10421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:51.251562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.306442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836f80a-7778-4614-8a68-93c694f486fe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.320447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/885] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:51.320447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009361-23c3-4a78-872a-463262a10421 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.355462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.356395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.356855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.357137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.357940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] RBAC: Adding query filter params (interface=public, service_id=2d4427c51f3d4be7accfe097cc269b17, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.358255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.359736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.368170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/886] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/endpoints?service_id=2d4427c51f3d4be7accfe097cc269b17&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.373331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.429219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6009361-23c3-4a78-872a-463262a10421 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.430857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.465795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.466616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.466903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.467180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.467841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.469265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.480774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d60f9-3959-48be-820d-2ddb61d09924 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:51.481345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:51.482515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:51.483345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/887] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:51.484149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.502156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.525209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.526034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.526315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.526584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.538822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:51.547595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:51.548866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:51.549763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/888] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:51.550647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.597911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3aa3d-0031-46a8-b6a6-fcec382d8906 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.632997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.633586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.633873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.634152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.634847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.636247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.652441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/889] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 02:56:51.652441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.674891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756785c-66be-4d6f-9cdd-ca751ccdda7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.696345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.715468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.716159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.716474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.716757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.717485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.717791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.719199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.729556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/890] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.730560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.776982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:51.798915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ee43-825c-47c7-a22e-be467a28bd39 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.838584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.839420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.839742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.840018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.841583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.841583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.842509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8c53a-bd4b-4e23-9567-39eea926fe15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.849522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf31339-5eb9-4084-bc11-0c8270634e26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:51.852731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/891] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.854079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1f53-464e-46a8-86da-795da81cf374 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.854373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1f53-464e-46a8-86da-795da81cf374 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.854652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1f53-464e-46a8-86da-795da81cf374 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.856052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/892] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:51.861219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.861511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.862230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.879409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:51.880638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:51.892468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:51.893132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:51.893421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:51.893706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:51.894463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:51.894734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:51.896124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:51.905199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/893] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:51.905199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:51.981469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/894] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:51.993293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.020295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.050284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46339a59-1c08-4200-8356-2c198aa3dc23 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.064177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.090415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.091256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.091544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.091819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.103702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.105206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.119305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c897e4d-729a-4688-b3dd-a78140e57ef4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:52.124160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:52.125133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:52.126679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:52.126679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:52.126679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:52.126679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:52.126934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/895] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:52.132649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.166010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.166890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.167154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.167447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.168113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.169545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:52.182294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:52.183255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:52.184102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/896] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jun 21 02:56:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:52.193411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.269658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ff0-1c93-4569-8f98-c81524006f61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.284601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.300350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369345d-d1fd-4844-ad85-ef632728b753 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.308658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.309639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.310101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.310590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.311568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:52.312055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.313689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.327505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/897] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:52.336460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.340640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.341537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.341967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.342385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.343315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:52.343790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.345576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.356585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/898] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:52.357738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.359256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.449765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de458bc-4250-45ed-9b45-8e2ce347288e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.475622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98188c70-6439-4285-add6-7097b853fd81 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.491837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.492837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.493272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.493643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.501871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.503950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.512035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.512794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.513080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.513406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.514181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:52.514489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.515931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:52.520917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:52.521861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:52.522826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:52.523350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/899] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:52.523421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/900] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:52.527981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.533413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.544861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.612022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.638257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf665f53-e98b-49b0-944f-13c8cbf2d709 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.639850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd764b-855b-4a07-86a1-f8169c034d35 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.676330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.677044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.677141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.677401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.677607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.678413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] RBAC: Adding query filter params (interface=public, service_id=77ca6055fce640dd8ddd09cb0feb53eb, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:52.678681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.678782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.679241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.679756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.680104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.681091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:52.681616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.683180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.686983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/901] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/endpoints?service_id=77ca6055fce640dd8ddd09cb0feb53eb&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:52.691433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.710358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/902] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:52.712059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:52.789931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:52.824100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b2f5-9d7b-48ad-ba32-64c6075effe4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.824814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c7034-e073-4e3b-ac6f-49e2da3bde53 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:52.863185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.864229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.864842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.865440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.867728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:52.868530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:52.869083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:52.869368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:52.870120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.872520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:52.880542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/903] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 02:56:52.951473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:52.953274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.030316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.066685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/904] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:56:52 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:56:53.072732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.075952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.189432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf3985-5aac-4922-8b0c-8502b234b40f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:53.224669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:53.225481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.225772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.226053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.236917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:53.238395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:53.256825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:53.257757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:53.258634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:53.258634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:53.258634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:53.258634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:53.258634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/905] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:53.263680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.365662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afbec4-4ae7-4bbd-a7b9-00348ac826ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:53.402540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:53.403222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.403510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.403789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.404586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:53.404902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:53.406302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.418575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/906] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:53.423851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.483361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.532428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74555b-b902-458c-9a95-db03f14288e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:53.567071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:53.567936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.568226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.568536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.575766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:53.577278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a1a2da8-7c30-4bd6-81c2-a6cfbdd9708b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:53.591793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:53.592854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:53.593803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/907] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:53.599932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.632549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.707062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188a66-579b-4ca8-8f6b-9caa743bcba0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:53.734264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458b73-cf7a-4808-9ef6-79cf504bdd47 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.734820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458b73-cf7a-4808-9ef6-79cf504bdd47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.734820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458b73-cf7a-4808-9ef6-79cf504bdd47 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.736328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/908] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:53.741375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.741891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.742380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.749791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:53.751130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.751619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.752093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.753295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:53.753843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:53.755603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.761017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:53.762224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:53.769959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/909] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:53.780535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.850554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/910] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:53.873124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:53.905724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff921-1d07-4c93-993d-00c14e916382 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:53.908801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.928323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:53.947923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:53.949238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:53.949804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:53.950252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:53.951522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] RBAC: Adding query filter params (role.id=726dc0d8fe874f0f9c4201e2d3091c9b, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=41a3f61f1038445aa7ab1224445fb2e3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:53.952073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:53.953818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:53.971024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/911] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/role_assignments?user.id=41a3f61f1038445aa7ab1224445fb2e3&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=726dc0d8fe874f0f9c4201e2d3091c9b => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:53.991156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f48d6-1918-47c5-887e-01f6020a4ace None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:54.030514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:54.031356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.031641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.031921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.032703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:54.034271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b8a60548-09e7-4ebc-a1a3-9fc85d6921c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:54.047626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:54.048814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 02:56:54.049781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:54.054504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/912] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:56:54.056116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:54.171635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532520bf-c242-4b16-8402-89a500c23bda None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.171930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532520bf-c242-4b16-8402-89a500c23bda None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.172238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532520bf-c242-4b16-8402-89a500c23bda None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.173954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/913] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:54.175826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0b079-9619-445e-85b1-c24b8d887d7c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:54.181768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.182315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.182815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.199840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:54.201509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:54.211898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:54.212610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.212893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.213166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.213889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:54.214445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:54.215793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:54.225646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/914] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:54.229848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:54.277634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:54.284432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/915] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:54.302471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:54.350254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:54.365244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912d11-dc02-4010-8837-d64d97008069 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:54.416748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:54.418133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.418854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.419741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.420989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:54.421754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:54.424003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:54.425466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777f030-f52b-4686-b218-75d7a0ecbf77 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:54.440342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/916] 10.222.0.16 () {56 vars in 1114 bytes} [Tue Jun 21 02:56:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:54.442157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:54.464875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:54.465551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.465834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.466107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.466852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:54.468228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2baa1-2744-4e58-b2ec-d90b7a19e1e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:54.488265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/917] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:56:54 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 02:56:54.539693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:54.558472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30dba9-27aa-4b86-b5f8-b86d43de4af2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:54.604798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:54.605481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:54.605769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:54.606040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:54.606793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:54.608308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec829b-ab45-4c9b-9de6-b04c877a985a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:54.634649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/918] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:56:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:55.002675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ccab-5e79-4d19-a2fb-587698391167 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.003416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ccab-5e79-4d19-a2fb-587698391167 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.003416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ccab-5e79-4d19-a2fb-587698391167 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.004928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/919] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:55.010213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.010637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.010815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.028031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:55.029279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:55.103684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/920] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:55.126673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87ccab-5e79-4d19-a2fb-587698391167 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:55.244127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:55.249161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87ccab-5e79-4d19-a2fb-587698391167 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:55.289530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:55.290269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.290550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.290821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.291485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:55.292914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:55.305547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:55.306533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:55.307550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/921] 10.222.0.16 () {56 vars in 1132 bytes} [Tue Jun 21 02:56:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:55.312710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:55.378943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:55.417853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75a9f-5fc2-40da-b94e-58940832dde6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:55.452924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:55.453652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.453940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.454218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.454938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:55.455397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:55.457149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:55.462332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/922] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:55.469498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:55.571953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e3ceb-7813-4202-9c76-48d82e184f10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:55.605856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:55.606527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.606812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.607089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.607803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:55.608111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:55.609534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f93ed-fcaa-4914-aa70-eb4c629548af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:55.615189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/923] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:55.621557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:55.730560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235f63-cf1b-4c1e-aaa0-79b7abd88452 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:55.757708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:55.766736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:55.767470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:55.768196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:55.768629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:55.769578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:55.770975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0fc26-b376-4787-8930-81362dae2103 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:55.795106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/924] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:56:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:56:56.133510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b75e31-02a0-4b94-91b5-7e550bf0f1b9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.134060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b75e31-02a0-4b94-91b5-7e550bf0f1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.134351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b75e31-02a0-4b94-91b5-7e550bf0f1b9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.136121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/925] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:56.141187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.141684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.141839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.160911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:56.162113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:56.235745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/926] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:56.258744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b75e31-02a0-4b94-91b5-7e550bf0f1b9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:56.288541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:56.365474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b75e31-02a0-4b94-91b5-7e550bf0f1b9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:56.399407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:56.400154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.400562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.400844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.411847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:56:56.420043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:56.421036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:56.421887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/927] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:56:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:56.429140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:56.532279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557101df-c0bf-4a9b-9802-9b47445f96dc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:56.542334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:56.570569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:56.571632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.572117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.572612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.573558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:56.574053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:56.575648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ebd09-28fe-491e-ba31-6fb44734f087 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:56.584743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/928] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:56:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:56.590574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:56.678492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:56.688466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a38ba-ba62-41f5-a79f-63c086143c08 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.688771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a38ba-ba62-41f5-a79f-63c086143c08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.689057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a38ba-ba62-41f5-a79f-63c086143c08 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.690871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/929] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:56:56.695348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c22355-aa6d-41ed-bbde-525038e054e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:56.695823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243b2f-856a-4573-b6bb-74c5b0b6f293 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.696108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243b2f-856a-4573-b6bb-74c5b0b6f293 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.696383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3243b2f-856a-4573-b6bb-74c5b0b6f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.713241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3243b2f-856a-4573-b6bb-74c5b0b6f293 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:56.714440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3243b2f-856a-4573-b6bb-74c5b0b6f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:56.732841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:56.733811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.734372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.734712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.748168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:56.749736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d9ee5b37-9c8e-4b0c-afb5-624892566f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:56.778215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:56.779212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:56.780113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:56.780113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:56.780113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:56.780113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:56.780113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/930] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:56.782030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a024-0f1c-4b05-9f04-f190bf706d00 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.782463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a024-0f1c-4b05-9f04-f190bf706d00 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.782881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a024-0f1c-4b05-9f04-f190bf706d00 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.788615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/931] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:56.788615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45a024-0f1c-4b05-9f04-f190bf706d00 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:56.804435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/932] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:56.804435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.804435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.804435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.824047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:56.825740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:56.900311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/933] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:56.910396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45a024-0f1c-4b05-9f04-f190bf706d00 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:56.913156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:56.950001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:56.953234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:56.953942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:56.954260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:56.954509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:56.955795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:56.956056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:56.957698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:56.964957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:56.975278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/934] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:56.976056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.027415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d657b-9a6d-48fb-9e20-5b2f27629dc2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.064980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.066208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.066497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.066807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.067476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.068880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:57.081377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:57.082635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:56:57.083702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.084782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/935] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:56:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:57.084975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.087668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdc18-f20b-42af-9ebb-81bceabab4b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.130304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.131437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.131898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.132369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.133688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.136178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.137680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING keystone.server.flask.common [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 02:56:57.164071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 02:56:57.164071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 02:56:57.164071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [parameters: {'id': 'd32572599b66433cb577320d1a60bb08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 02:56:57.164071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d32572599b66433cb577320d1a60bb08.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d32572599b66433cb577320d1a60bb08. Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:56:57.171578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 21 02:56:57.172437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jun 21 02:56:57.173413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 02:56:57.175836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 02:56:57.177873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:56:57.179917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': 'd32572599b66433cb577320d1a60bb08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.182976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:57.184966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d32572599b66433cb577320d1a60bb08. Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/936] 10.222.0.16 () {60 vars in 1140 bytes} [Tue Jun 21 02:56:56 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:57.187121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.216305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.226127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4aaee-903c-4359-bef7-edae25085824 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.242267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.269667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.270641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.270935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.271221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.279536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.281067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.292876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec90e9-7acb-47e7-b3f0-dd15bf4e6c20 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:56:57.296670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:57.297589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:56:57.298540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.302901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/937] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:57.303779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.328625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.329332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.329618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.329899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.330865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:57.331178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.332794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.340647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/938] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:57.341444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.417530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f23827-9c1e-4e81-a543-0a62a4e07014 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.448139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2faae4-b49d-436b-a8bc-018422d4317a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.454239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.455051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.455420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.455995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.463334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.464832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.467557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:57.479176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:57.480095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/939] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:57.481435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.492275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.493028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.493316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.493594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.494452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:57.494768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.496205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.498362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.515767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/940] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:57.517630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.597787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0a1ce-6a46-47a0-8d6f-dbe8d329e59b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.624870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0b59c-e30b-4bc8-9e20-579b3bbc2f40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.636558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.637229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.637517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.637810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.638531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:57.638840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.640523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.646091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/941] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:57.646897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.665323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.666012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.666296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.666561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.667323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:57.667636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.669126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.696453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/942] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:57.696453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.741378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.751517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:57.778100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815ea04-7b18-4eda-a52b-220c74893c27 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.817071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.818163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.818594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.818967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.821350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebda4b-c2b7-44cd-8ed0-8390299fb624 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:57.835339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.836893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-570818c8-0933-4d27-848c-2d428eb7ba61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:56:57.860185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:57.861432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/943] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:57.862888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3a58c-8cd2-4cb0-a5df-88de6099dfc3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.863258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3a58c-8cd2-4cb0-a5df-88de6099dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.863703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3a58c-8cd2-4cb0-a5df-88de6099dfc3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.868610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/944] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:57.869149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3a58c-8cd2-4cb0-a5df-88de6099dfc3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.872635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:57.874264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.874551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.874834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.876014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] RBAC: Adding query filter params (interface=public, service_id=abff6002bf0043fcae95dda2d73c7b54, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:57.876336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:57.878507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbafc68-3877-41b2-9dd7-d653d6ea0ed9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:57.885340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/945] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/endpoints?service_id=abff6002bf0043fcae95dda2d73c7b54&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:57.886765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:57.887119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:57.887409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:57.907287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:57.908918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:57.985453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/946] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:56:57.986392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:57.990032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3a58c-8cd2-4cb0-a5df-88de6099dfc3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.023577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.027795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.035203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.036016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.036646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.036765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.037564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:58.038017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.039520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.056944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/947] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:58.056944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.105887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced7a33-94c6-4fa9-b945-19775cfdaa80 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.142235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.143093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.143640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.143744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.144360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.147974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.154318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/948] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:56:57 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 02:56:58.154988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.178226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be12ebd-6e7f-4fe6-a6c3-20a528f9bf72 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.231663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.233360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.234596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.234888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.237715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.238484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:58.256779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:58.258156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:56:58.259233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:56:58.266512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/949] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:58.267632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.301913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.314374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d497b-a386-4017-8f1c-b0c5bdfbd52d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.334510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.359851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.360898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.361489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.361489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.369494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.370953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:56:58.387520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:56:58.388456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:56:58.389370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:56:58.389804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3b18-95e0-45f3-94b1-b73d39b95ef9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.389894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/950] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:56:58.396022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.433918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.435099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.436355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.437367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.438391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:58.440920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.443542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.454084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/951] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:58.458357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.521422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9296e7-72c1-4192-859f-c799c6aa158a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.559406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.560316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.560790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.561216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.562197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:58.562688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.564322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.586823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/952] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:56:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:56:58.588344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:56:58.591268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477c7be-32a6-47df-9894-48ca240d0249 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.604991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.613330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:56:58.635885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.636862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.637307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.637725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.638904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:56:58.639766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.641624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa8cde-8ad4-4aff-8e15-69a3cddae0e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.648749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/953] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:56:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:56:58.715749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109821f-1577-465f-8f8e-aaf4feddc008 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:56:58.756032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:56:58.756988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:58.757275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:58.757572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/41a3f61f1038445aa7ab1224445fb2e3/roles/726dc0d8fe874f0f9c4201e2d3091c9b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:58.759912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=41a3f61f1038445aa7ab1224445fb2e3, role_id=726dc0d8fe874f0f9c4201e2d3091c9b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:56:58.761411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651c4eb-975a-4588-94e2-3167b4a63586 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:56:58.780205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/954] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:56:58 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/41a3f61f1038445aa7ab1224445fb2e3/roles/726dc0d8fe874f0f9c4201e2d3091c9b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 02:56:59.803834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d86e8-aaaf-4412-a81c-e127950aeb04 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:59.804433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d86e8-aaaf-4412-a81c-e127950aeb04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:59.804433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d86e8-aaaf-4412-a81c-e127950aeb04 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:59.806011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/955] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:56:59.811464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c94cd-670c-4860-8037-6f026283155e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:56:59.811876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c94cd-670c-4860-8037-6f026283155e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:56:59.812054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c94cd-670c-4860-8037-6f026283155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:56:59.829350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-397c94cd-670c-4860-8037-6f026283155e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:56:59.830614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-397c94cd-670c-4860-8037-6f026283155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:56:59.901637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/956] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:56:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:56:59.927086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d86e8-aaaf-4412-a81c-e127950aeb04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.034402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d86e8-aaaf-4412-a81c-e127950aeb04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.068315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.068945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.069230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.069508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.070224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.071676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.073032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 02:57:00.088489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 02:57:00.088489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 02:57:00.088489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [parameters: {'user_id': '0bd48f3a4e574f02a2bd318a1c930ffe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 02:57:00.088489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1576752a-e4c7-4ac9-a200-8a670cd90c9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 21 02:57:00.105637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:57:00.106552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 02:57:00.108180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 02:57:00.109159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 02:57:00.110537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 02:57:00.113658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'user_id': '0bd48f3a4e574f02a2bd318a1c930ffe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:00.115648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 21 02:57:00.117032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.118151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/957] 10.222.0.16 () {60 vars in 1131 bytes} [Tue Jun 21 02:56:59 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:00.118646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c94cd-670c-4860-8037-6f026283155e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.124135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:00.220317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c94cd-670c-4860-8037-6f026283155e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.255892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.256741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.257023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.257345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.264994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.266454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:57:00.282283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:00.284793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:00.287096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/958] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:00.288506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ab12e-029f-4361-bd28-eb8c9788dd0c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.288506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ab12e-029f-4361-bd28-eb8c9788dd0c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.288506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ab12e-029f-4361-bd28-eb8c9788dd0c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.292382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.292591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/959] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:57:00.298249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.298788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.299355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.317073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:00.319046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:00.361647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:00.363643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:00.402814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/960] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:00.409627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e2735-5f46-45c9-9179-da6b92475fb4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.426258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.447064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.448122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.448641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.449198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.450259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:00.450805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.452819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.492457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/961] 10.222.0.16 () {56 vars in 1101 bytes} [Tue Jun 21 02:57:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:00.541879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789da3fc-14c0-4ce1-b52e-de4a51d5fc3c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.582345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.583088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.583374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.583647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.584325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.585900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:00.599500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:00.600708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:00.601571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/962] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:57:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:00.607304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.717703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb16f-0e24-47d9-8f18-ea1001b1aaa3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.743803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:00.757339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.758023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.758311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.758590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.759309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:00.759625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.761152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3792d-c9db-4fd8-8429-a27b3b410e53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.766568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/963] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:00.772870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.778620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47723e-8802-4a8c-b4c3-44d1605e5218 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.778919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47723e-8802-4a8c-b4c3-44d1605e5218 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.779203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47723e-8802-4a8c-b4c3-44d1605e5218 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.780804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/964] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:00.785861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.786150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.786427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.802863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:00.804055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:00.816669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:00.881082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/965] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:57:00.889559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c8275-605f-49f6-bcc4-11a961910d4d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:00.903630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:00.931422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:00.932362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.933085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.933424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.934498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:00.935089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:00.937331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e70816-3f29-49f5-bdd0-672413549e16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:00.943741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/966] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:00.946146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8815d-b56c-416d-9d55-a5ae4c4b102a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:00.946433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8815d-b56c-416d-9d55-a5ae4c4b102a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:00.946709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8815d-b56c-416d-9d55-a5ae4c4b102a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:00.947967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/967] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:00.952282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8815d-b56c-416d-9d55-a5ae4c4b102a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.012629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f560dec-8869-4539-81e6-433b6c4da993 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.028637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:01.047015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.047690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.047975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.048309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.049141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:01.050523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4dc87-5de3-4b2a-af8c-ccd59cc9dc72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:01.054960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8815d-b56c-416d-9d55-a5ae4c4b102a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.056688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:01.065702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/968] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:57:00 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:01.076678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.076678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.076678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.087445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:01.088740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:01.102411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.103161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.103456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.103740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.104576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:01.106143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:01.144550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/969] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:57:00 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:01.187670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/970] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:01.211828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.333400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eb608-49b1-49ce-9400-dcc9533d7199 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.368042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.368916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.369202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.369437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.380479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:01.388763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:01.389713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:01.390573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/971] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:57:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:01.397993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.503730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18101514-ea7f-42fc-b571-fe0741c936a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.505314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:01.540517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.541192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.541473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.541747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.542494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:01.542799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:01.544172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:01.557074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/972] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:57:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:01.558702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.577375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:01.667178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518193-b639-49d2-bba4-5757ccef76fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.701325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.702125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.702405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.702673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.713782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:01.715246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:01.733259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:01.734154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:01.739906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:01.739906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:01.739906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:01.739906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:01.739906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/973] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:01.741719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.846662 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8719b82-22e3-480e-9381-1c28cc6461db None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:01.882595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:01.883237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:01.883535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:01.883804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:01.884589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:01.884891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:01.886268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:01.898539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/974] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:01.904095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:01.972740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:02.007450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e096818-ef9c-4862-80ce-626c83d4c403 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:02.041261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:02.042083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:02.042369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:02.042644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:02.050341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:02.051801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:57:02.066254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:02.067214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:02.068139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/975] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:57:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:02.074745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:02.101452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:02.178487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce64b-3952-4e55-b019-82a17ca0353e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:02.215871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:02.216607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:02.216898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:02.217177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:02.218035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:02.218347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:02.219803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:02.242370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/976] 10.222.0.16 () {56 vars in 1149 bytes} [Tue Jun 21 02:57:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:02.243962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:02.351734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a4f2c-c558-4d12-83e1-f1c371146024 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:02.377777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:02.387254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:02.387909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:02.388196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:02.388498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:02.456983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:02.458455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:02.564935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/977] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:02 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:02.566269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:02.651705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:02.672745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6c76f-df0c-447a-a94a-4773bc169990 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:02.708509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:02.709323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:02.709595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:02.709876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:02.720677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:02.722220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:02.740435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:02.741393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:02.747067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:02.747067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:02.747067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:02.747067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:02.747067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/978] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:02.748902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:02.777530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:02.852268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a6fc-b74b-4e19-9c06-1a27c5988cba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:02.887806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:02.888510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:02.888795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:02.889076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:02.889811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:02.890125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:02.891512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:02.904121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/979] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:02.910180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.016218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468ad3c-413c-44b3-bdfc-969eccbc9a62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.050902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.052030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.052487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.052931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.055131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.060940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.062599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8c3cb948-132b-4259-bcbe-0c2f3aebe7cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:57:03.077883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:03.078845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:03.079753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:03.086526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/980] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:03.087760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.105750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189143-14c3-4ee0-a3ce-543b81efcd27 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.105963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189143-14c3-4ee0-a3ce-543b81efcd27 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.106241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189143-14c3-4ee0-a3ce-543b81efcd27 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.107650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/981] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:03.112930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc862ae2-5436-4f2b-b803-43ca8da4b19e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.113217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc862ae2-5436-4f2b-b803-43ca8da4b19e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.113499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc862ae2-5436-4f2b-b803-43ca8da4b19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.131013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc862ae2-5436-4f2b-b803-43ca8da4b19e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:03.132187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc862ae2-5436-4f2b-b803-43ca8da4b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:03.187420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.202667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df022f65-3e7e-4abf-8554-e9567c8853fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.209941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/982] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:03.211357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cd610-d84d-4369-bfeb-0e2e11c7c24b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.211730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cd610-d84d-4369-bfeb-0e2e11c7c24b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.211938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cd610-d84d-4369-bfeb-0e2e11c7c24b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.213342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/983] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:03.218493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.218778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.219054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.235345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:03.236637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:03.238411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.239096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.239386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.239666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.240489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.240804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.242249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.255610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/984] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:03.256437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.306647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/985] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:57:03.307495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.363157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.363346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65974ffd-6df5-44ea-952c-f33b7a15ffc1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.397911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.398714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.399001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.399278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.399944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.401378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.408700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b113a-717e-47c4-b8f8-f6ad68baa451 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:03.413157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:03.414145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.414992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/986] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:03.415811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.428278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.445122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.445789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.446072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.446348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.447186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] RBAC: Adding query filter params (role.id=726dc0d8fe874f0f9c4201e2d3091c9b, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=41a3f61f1038445aa7ab1224445fb2e3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.447498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.448985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.455788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/987] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/role_assignments?user.id=41a3f61f1038445aa7ab1224445fb2e3&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=726dc0d8fe874f0f9c4201e2d3091c9b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:03.456642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.529134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4a3b3-4149-4899-9c53-f39ff0009441 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.570792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.572293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.572614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.572941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.573653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.573922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf0ea7-6bf8-4e4f-a4b4-8960bb3b4b60 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.575099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:03.592785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:03.593877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:03.595298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:03.596326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.596405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/988] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:03.600707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.615719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.616319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.616622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.616900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.617616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.617933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.619313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.625762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/989] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:03.631925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.704358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.736733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c9bb9-1d23-41a7-863b-a026a7b278eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.742727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a595966-7396-47a2-9f0b-e7495820a3ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.773854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.774533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.774818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.775093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.775809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.776117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.777275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.777554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.777999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.778287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.778563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.779265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.779573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.780953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.783707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/990] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:03.788881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.790185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/991] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:03.795620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.842769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:03.897141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadce17-eff3-4633-9d8d-4e59a21e9aca None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.897484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4050b2-b92a-4d03-a907-661a71ae092f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:03.931600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.932251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:03.932799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.932799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.932967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:03.933039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.933107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:03.933107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:03.933603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] RBAC: Adding query filter params (interface=public, service_id=2cb68832afda4a0c954576aaa375bc7d, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.933803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:03.933897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.934111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:03.935263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.935490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:03.941098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/992] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/endpoints?service_id=2cb68832afda4a0c954576aaa375bc7d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:03.944585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/993] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:03.946332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.946877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:03.949644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:04.054565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371f63-9d0c-4a3f-99dd-95e90a3eeb26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:04.057078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3b11-d03e-4cfe-98d6-a9280f7bb244 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:04.066666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:04.091719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:04.092398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:04.092733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:04.093012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:04.093700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:04.093869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:04.094505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:04.094788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:04.095062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:04.095156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:04.095924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] RBAC: Adding query filter params (interface=admin, service_id=abff6002bf0043fcae95dda2d73c7b54, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:04.096535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:04.098216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc005c4c-df6d-43a4-8dc3-e50d7908b81f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:04.107748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/994] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 02:57:04.108317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/995] 10.222.0.16 () {56 vars in 1249 bytes} [Tue Jun 21 02:57:03 2022] GET /identity/v3/endpoints?service_id=abff6002bf0043fcae95dda2d73c7b54&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:04.109336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:04.226922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091902-42a3-45df-a9db-aa6cd5c7ac83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:04.264809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:04.265509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:04.265782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:04.266057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:04.266752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:04.268301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbba65-ecf3-4938-ad18-bf2510596510 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:04.275141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/996] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:57:04 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 02:57:05.494850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4f92-7365-4be0-98c7-ee74c9f58c22 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:05.495398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4f92-7365-4be0-98c7-ee74c9f58c22 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:05.495398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade4f92-7365-4be0-98c7-ee74c9f58c22 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:05.497022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/997] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:05.502621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:05.503067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:05.503319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:05.520687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:05.521858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:05.553080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:05.593208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/998] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:05.615139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade4f92-7365-4be0-98c7-ee74c9f58c22 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:05.687263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:05.724877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade4f92-7365-4be0-98c7-ee74c9f58c22 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:05.759651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:05.760368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:05.760677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:05.760947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:05.761597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:05.762978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:05.775473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:05.781436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:05.782321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/999] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jun 21 02:57:05 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:05.784475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:05.890719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ec518-f9d6-41e5-a946-73a7632df5c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:05.925535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:05.926206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:05.926490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:05.926766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:05.927477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:05.927781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:05.929180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2077d-6437-43e3-895e-68a9adf1a4cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:05.934295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/1000] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:05.939895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.045200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76141f-54b4-4c73-aee7-34d0317d9beb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.074700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.082227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.082910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.083267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.083557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.084259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:06.084614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.086229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98c52a-813a-49e6-8a77-24652483164b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.091141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/1001] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:06.200245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f45ff-1002-4f1d-90df-774a128a48c2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.200630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f45ff-1002-4f1d-90df-774a128a48c2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.200833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f45ff-1002-4f1d-90df-774a128a48c2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.202469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/1002] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:06.209090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.209457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.209646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.229258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:06.230489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:06.291065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b51b-7b28-457f-892a-de06a161dd2f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.291391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b51b-7b28-457f-892a-de06a161dd2f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.291630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b51b-7b28-457f-892a-de06a161dd2f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.293078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/1003] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:57:06.298317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.298543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.298832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.311305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/1004] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:06.315632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:06.316867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:06.333749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.344469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.393749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/1005] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:57:06.416451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.443253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d031ee-c073-43d8-adcb-5fee44a2b934 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.471376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.484793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.485600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.485940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.486219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.487028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.488627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:06.501290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:06.503000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:06.503846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/1006] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:06.509232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.532948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8d2e-06ba-4c29-a54a-dd28cb2c8d8b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.570273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.571017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.571361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.571641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.572363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.573778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:06.586320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:06.587325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:06.588219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/1007] 10.222.0.16 () {56 vars in 1126 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:06.593442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.620195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff385a2-d7fd-4d63-ba1a-ac3610b02e3b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.621340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.660178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.661673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.663888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.663888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.663888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:06.663888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.665369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.672562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/1008] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:06.678395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.707210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e4160-b4f3-45c7-83e5-39f65d83ac6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.751328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.753260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.753954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.754244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.754522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.755296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:06.755606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.757021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.764766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1009] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:06.770860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.808384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ad0d4-8b48-420f-b46a-aeb6ccd7c681 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.845400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.846057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.846344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.846622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.847329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:06.847638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.849084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.854863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/1010] 10.222.0.16 () {56 vars in 1112 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:06.860820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.869511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:06.880445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e77d3-ff9b-44f1-a75b-044c482276da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.917192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:06.917787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:06.918066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:06.918339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:06.919052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:06.919361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:06.920863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:06.926463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Tue Jun 21 02:57:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:06.931980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:06.975833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f3b8a-ae36-4c9f-9af4-504b3a7976be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:06.992223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:07.011621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:07.012374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:07.012702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:07.012979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:07.013728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:07.015113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960d2a4-31c2-4209-90ed-df6e8ceb4433 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:07.034959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/1012] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:57:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:07.048621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd1e72-edc3-46ee-8b14-15eeac695b72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:07.099054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:07.099633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:07.099916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:07.100188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:07.100976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:07.102365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f466639-dc2f-4026-9a80-89f354ce261e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:07.128457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:57:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 02:57:08.158457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734d30-cc21-47af-912e-1a8d56da4bc6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:08.158999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734d30-cc21-47af-912e-1a8d56da4bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:08.159305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734d30-cc21-47af-912e-1a8d56da4bc6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:08.160639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1014] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:08.166665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d2af-1cd1-483e-b6b7-4f7a049c9176 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:08.167038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d2af-1cd1-483e-b6b7-4f7a049c9176 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:08.167293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d2af-1cd1-483e-b6b7-4f7a049c9176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:08.185293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1d6d2af-1cd1-483e-b6b7-4f7a049c9176 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:08.186476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1d6d2af-1cd1-483e-b6b7-4f7a049c9176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:08.190526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:08.268034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1015] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:08.291143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a734d30-cc21-47af-912e-1a8d56da4bc6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:08.398468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a734d30-cc21-47af-912e-1a8d56da4bc6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:08.431743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:08.432350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:08.432692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:08.432998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:08.433739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:08.435126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2786c5-a675-4c76-af5a-d2317066581c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:08.460441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1016] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:57:08 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:09.167466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246f36-0af6-4832-af35-3e60e3e6dd05 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.167932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246f36-0af6-4832-af35-3e60e3e6dd05 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.168009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0246f36-0af6-4832-af35-3e60e3e6dd05 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.169534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1017] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:09.175155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.175594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.175771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.193830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:09.195022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:09.221881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5bc6b-4394-4bbf-80db-fafa1e916896 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.222196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5bc6b-4394-4bbf-80db-fafa1e916896 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.222451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5bc6b-4394-4bbf-80db-fafa1e916896 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.223064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:09.224026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1018] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:57:09.229155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.229454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.229744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.249882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:09.251562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:09.276326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1019] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:09.300614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.320591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1020] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:57:09.344288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.413341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d229d-8ef2-473b-91a2-c66f7afb0d9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:09.414064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:09.449412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:09.450133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.450419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.450699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.451959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:09.452876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e8353-b3a8-48b9-8c36-e0e8164f3868 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:09.453448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:09.456451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:09.460713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.488840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:09.496445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:09.497280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.497565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.497840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.508737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:09.516888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:09.518129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:09.519173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1022] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:57:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:09.524204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.570567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117fe8f-ad15-4382-888c-c08d61561184 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:09.608096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:09.609127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.609416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.609648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.610660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:09.612300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:09.634559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4780f9-b526-4633-960b-4834c827314f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:09.648456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1023] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:57:09 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 02:57:09.663255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:09.673657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:09.674544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.674832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.675112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.676029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:09.676393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:09.677959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:09.693659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1024] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:57:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:09.695073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.785535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:09.813315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527d77e-cf92-42c9-b57e-cfe44ae9d52c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:09.850856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:09.851694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:09.851976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:09.852251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:09.863575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:09.865081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:09.883538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:09.884528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:09.885426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:09.885426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:09.885426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:09.885426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:09.885426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1025] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:09.891615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:09.995989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23684e-7661-49ee-8575-450277d2d2f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.030395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.031065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.031347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.031622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.032347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:10.032684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.034083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.046537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1026] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:10.052164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.166546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91692b3d-15c6-4cb9-8546-f096cc68e0fe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.196479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.203594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.204601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.204885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.205275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.218183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.219092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:57:10.236988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:10.238001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:10.238952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1027] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:10.243982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.294997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.348137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe212-cec0-4ca7-bd5e-bfb42bb2eb0e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.382457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.383146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.383429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.383710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.384587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:10.384886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.386322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05026a54-5977-45a3-a27b-be9ae7b813f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.404502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1028] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:10.410312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.515953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a20dcd-d6f7-45fa-ab9c-850c05ef0c2c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.535841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7940-9f62-478c-92c6-d986676cce64 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.536219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7940-9f62-478c-92c6-d986676cce64 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.536615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a7940-9f62-478c-92c6-d986676cce64 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.538341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1029] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:10.544132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.544513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.544861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.550840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.551499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.551784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.552063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.561970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:10.563189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:10.623875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.636138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.637651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.640456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1030] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:10.661493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.695995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.747006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1031] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 02:57:10.753866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.774007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463c44d-8c66-40d0-9aab-9c7645ef7cad None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.814606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.815833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.816193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.816505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.817180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.818578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:10.831014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:10.832005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:10.832895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1032] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:10.838195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.855632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.866306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0213-31a9-4f82-86ea-abe2f6bed8b5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.901452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.902199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.902514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.902786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.914972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.916460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:10.934467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:10.935389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:10.936261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:10.936261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:10.936261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:10.936261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:10.936261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:10.941654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:10.947777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:10.951387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f6e5-9b85-49b2-b3dc-d7dd33c571e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:10.988825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:10.989615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:10.989972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:10.990320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:10.991116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:10.991498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:10.992917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:10.998259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1034] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:11.004055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.049185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532a8f6-243e-4edd-8a0a-c43a4db1fb64 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.083455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.084121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.084439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.084710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.085433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:11.085742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.087122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.098304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1035] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:11.103810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.106452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:11.115946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa122-2194-4523-93ff-303ee6281d62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.159620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.160223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.160559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.160843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.161570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:11.161886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.163798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.175566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:11.177117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.209801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:11.210056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70f665-bb81-48db-b12f-eccd62cee275 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.248095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.248943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.249234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.249625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.257133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.258618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:57:11.273514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:11.274483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:11.275385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1037] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:11.282245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.289644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786863f-5ceb-49d7-9077-bcb9eef27121 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.325339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.326016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.326304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.326586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.327382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] RBAC: Adding query filter params (interface=public, service_id=4e18376bda3b4951913fcbce5ec22c87, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:11.327702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.329131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.338973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1038] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/endpoints?service_id=4e18376bda3b4951913fcbce5ec22c87&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:11.340123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.354138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:11.400026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3fe5b-87eb-4198-a49d-159790d4a8b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.433076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:11.438392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.439065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.439344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.439669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.440476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:11.440788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.442234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.442749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef9dea-49bb-4de1-a846-d75c275232af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.454881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1039] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:11.461020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.478373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.479211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.479568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.480090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.480822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.482232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f41fe9-a09a-431a-b55a-9280d58041bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.489725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1040] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 02:57:11.569323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61816a2-d6d5-46c9-ba5e-5c21c15abe96 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:11.594891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:11.609907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:11.610862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.611372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.611837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.612925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] RBAC: Adding query filter params (role.id=726dc0d8fe874f0f9c4201e2d3091c9b, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:11.613398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:11.615034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f123bc-a034-4045-86f1-0d9cf6ed35ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:11.621733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1041] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=726dc0d8fe874f0f9c4201e2d3091c9b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:11.737336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940250d-ab3f-46cb-b0b0-7b3eb2b853e4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.737605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940250d-ab3f-46cb-b0b0-7b3eb2b853e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.737869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940250d-ab3f-46cb-b0b0-7b3eb2b853e4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.739542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1042] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:11.745262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:11.745588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:11.745784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:11.764635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:11.765856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:11.838143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1043] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:11.862590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940250d-ab3f-46cb-b0b0-7b3eb2b853e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:11.971513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940250d-ab3f-46cb-b0b0-7b3eb2b853e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:12.006251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:12.006979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:12.007312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:12.007549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:12.008777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:12.010219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797ed0f-573e-4553-a1e1-86808bfe6b39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:12.012059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:12.017678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:12.117655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ad378-7e7a-41c1-8b3a-66fe18fbd5d5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:12.134749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:12.166729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:12.167429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:12.167710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:12.167960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:12.168757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:12.170220 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd283b18-0aa3-43c0-83f5-45179e891c51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:12.199191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1045] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:57:12 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:13.524339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c306-cb60-484f-85dd-c86bf67d0982 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.525168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c306-cb60-484f-85dd-c86bf67d0982 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.525168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c306-cb60-484f-85dd-c86bf67d0982 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.526575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1046] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:13.531812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.532269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.532360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.549942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:13.551094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:13.620454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:13.637006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43bf4-32e0-4fa5-9cc5-649ab2e9cfa9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.637300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43bf4-32e0-4fa5-9cc5-649ab2e9cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.637581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43bf4-32e0-4fa5-9cc5-649ab2e9cfa9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.639098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1048] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:13.642143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:13.643846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.644141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.644453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.661231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:13.662858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:13.680983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:13.733120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1049] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:13.749052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07441-a3cc-4d1f-9039-9eac2eeff9e7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:13.751268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:13.756844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:13.793082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:13.793801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.794087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.794364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.795028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:13.796885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:13.808718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:13.809863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:13.810727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1050] 10.222.0.16 () {56 vars in 1105 bytes} [Tue Jun 21 02:57:13 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:13.816208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:13.863768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b8741-da30-4be4-9162-d3cbb52e4fb9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:13.899128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:13.899959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.900242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.900634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.911673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:13.919849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:13.920834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:13.921694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1051] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:57:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:13.926737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4196728-e7cb-44bb-a1dc-e46d81868dfe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:13.927258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:13.944209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:13.963259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:13.963937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:13.964224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:13.964531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:13.965252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:13.965563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:13.966964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:13.976877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1052] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:13.977958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.011582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.040899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624d772-9801-455f-af26-5db737fd3951 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.083192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.083791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.084082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.084246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e8c1-ccfd-409d-b652-53a1933a73ac None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.084364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.085162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:14.085448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.087163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.095308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1053] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:57:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:14.100992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.120497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.121177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.121583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.121774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.122440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:14.122752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.124507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.129821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1054] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:14.135581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.184891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.208874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20af56-e6ad-407d-b2d0-4aa49d6c344a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.244972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.245766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.246786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.247074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.247301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.254503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa3002-3d5c-4be9-8b54-8cef29e28766 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.261014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.262966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c35069c9-801a-4bb8-a2e1-36675edafa6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:14.284952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:14.285895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:14.286806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:14.286806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:14.286806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:14.286806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:14.286806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1055] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:14.290525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a38d-5a00-4b47-b092-8b4281918363 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.290873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a38d-5a00-4b47-b092-8b4281918363 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.291155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a38d-5a00-4b47-b092-8b4281918363 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.291957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.292719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1056] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:57:14.292836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.292932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.293209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.293434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a38d-5a00-4b47-b092-8b4281918363 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.293967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.295359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31214b6-0089-48d2-96a8-7b2dc160f598 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.314545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1057] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:57:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:14.316095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.316383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.316696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.335850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:14.337140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:14.417813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1058] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:14.418420 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a38d-5a00-4b47-b092-8b4281918363 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.441777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.447680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.463332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.464015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.464303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.464609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.465853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:14.466448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.468032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.483064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1059] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:14.488730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.533243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.555887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11814c-3044-4c04-b02f-10b5a4532f75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.592065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.592821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.593105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.593385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.596402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e35abf-2c0e-40ea-935a-7755c7651553 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.604102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:14.617643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:14.618741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:14.619742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1060] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:14.625040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.638962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.639973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.640440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.640857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.648893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.650540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:57:14.664710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:14.665730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:14.666674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1061] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:14.671481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.720204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.739524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd308f-381c-4341-9f9a-38157e6bd3c0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.776268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.776984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.777273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.777554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.778316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:14.778628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.779370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152702d-176b-4210-a8a7-69ae477a4608 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.780030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.782297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.791054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1062] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:14.796732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.822517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.823176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.823466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.823754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.824642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:14.824958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.826411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.846005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1063] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:14.851414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:14.910156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d09b1-f00f-41aa-bf4a-836217e74fe2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.946738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:14.947623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:14.947909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:14.948181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] PATH_INFO: `/v3/users/ca485756c8ed4d3580d3269c865c71ae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:14.963079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] RBAC: Authorizing `identity:get_user(user_id=ca485756c8ed4d3580d3269c865c71ae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:14.964536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d455b71-f0c3-422b-b0b6-de0eb0039c5e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:14.964647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:14.966246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:14.966374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1064] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/users/ca485756c8ed4d3580d3269c865c71ae => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:14.972602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.007532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.008758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.009237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.009735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.019877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.021358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.032240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:15.040934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:15.041883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:15.042782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:15.042782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:15.042782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:15.042782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:15.042782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1065] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:15.047704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.085997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb660c-a4d4-4218-a546-a6988bf5b036 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.124001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.124699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.124984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.125263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.167536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f8718-9947-4760-bb45-f1d5afb33608 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.196725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.198287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.206404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.207073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.207357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.207634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.208365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:15.208702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.210101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.222166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1066] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:15.227639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.245101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.261053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.313434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1067] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:15.319399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.340804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771086-4823-445a-9656-c3cd9c55a38a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.379167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.379898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.380182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.380486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.387503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.388990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:57:15.403764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:15.404783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:15.405716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1068] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:15.411063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.428778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3df25-7505-4e3f-be13-0a656d5588a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.466057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.467107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.467475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.467803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.469425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.471292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.473764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1069] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:15 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:15.479510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.495180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.507606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.525878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352487e-debb-4cac-94c5-e2cdce16de29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.563819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.564713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.565055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.565335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.566141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:15.566455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.567913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.585432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1070] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:57:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:15.586284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:15.592370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a98c6-2a87-48ec-9a91-7f3d62578775 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.631693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.632282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.632649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.633108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.633822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=e4d6900a26954f29afed8163cffc4632, user.id=ca485756c8ed4d3580d3269c865c71ae) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:15.634145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.635616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581c420-e83f-4ba6-96e7-85d184a0c013 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.641338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1071] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:15 2022] GET /identity/v3/role_assignments?user.id=ca485756c8ed4d3580d3269c865c71ae&scope.project.id=e4d6900a26954f29afed8163cffc4632&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:15.698340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc057c1-98c9-4824-aaf3-cb89d864d119 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:15.737393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:15.738323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:15.738615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:15.738889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/726dc0d8fe874f0f9c4201e2d3091c9b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:15.741274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=1c16671fd5844a77ac6e56dff36f86f8, role_id=726dc0d8fe874f0f9c4201e2d3091c9b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:15.742737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ae7ce-cd6a-429b-81d3-e6d8609657b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:15.746765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:15.760314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1072] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:57:15 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/1c16671fd5844a77ac6e56dff36f86f8/roles/726dc0d8fe874f0f9c4201e2d3091c9b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:57:16.374606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139aad6-86c1-41f4-b8c9-b7b3bd770da7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:16.374988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139aad6-86c1-41f4-b8c9-b7b3bd770da7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:16.375193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139aad6-86c1-41f4-b8c9-b7b3bd770da7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:16.377025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1073] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:16.382901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:16.383182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:16.383473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:16.400652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:16.401847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:16.478546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1074] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:16.501569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139aad6-86c1-41f4-b8c9-b7b3bd770da7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:16.607017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:16.626162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139aad6-86c1-41f4-b8c9-b7b3bd770da7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:16.666435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:16.667100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:16.667383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:16.667657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:16.756581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:16.757995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:16.857033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:16.872083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1075] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:16.879107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:16.994196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb109bb-9e6b-49d7-a795-30a5bcde860c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:17.025945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:17.040007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:17.040851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.041135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.041411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.053797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:17.055745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-daf90a64-d33a-46c2-b7b6-7efc2dd91848 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:17.073674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:17.082089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:17.083000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:17.083000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:17.083000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:17.083000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:17.083000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1076] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:17.084842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:17.207077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf645441-eed2-4dd0-b79e-36f088abe576 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:17.244184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:17.245046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.245390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.245746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.246614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:17.246993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:17.248390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfbf92-7f5f-4d7b-a0b9-14e33a16ee7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:17.262122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1077] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:17.792474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf672-3656-411c-82dc-261c4c346ff2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.792752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf672-3656-411c-82dc-261c4c346ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.793052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdaf672-3656-411c-82dc-261c4c346ff2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.794429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1078] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:17.799969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2911c7-67d0-4d5e-8f87-ec3d96cc2ba4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.800446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2911c7-67d0-4d5e-8f87-ec3d96cc2ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.800542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2911c7-67d0-4d5e-8f87-ec3d96cc2ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.819415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7d2911c7-67d0-4d5e-8f87-ec3d96cc2ba4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:17.820679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d2911c7-67d0-4d5e-8f87-ec3d96cc2ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:17.867564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:17.894230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1079] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:17.907787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d8bc-0a6d-40d1-aabc-08607248abdb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.908567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d8bc-0a6d-40d1-aabc-08607248abdb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.908567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00d8bc-0a6d-40d1-aabc-08607248abdb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.909858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1080] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:17.915308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:17.915601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:17.915856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:17.916262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00d8bc-0a6d-40d1-aabc-08607248abdb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:17.933943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:17.935133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:18.002053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1081] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:18.019722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00d8bc-0a6d-40d1-aabc-08607248abdb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.026517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.052985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.053702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.053981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.054252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.064324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:18.071992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:18.073065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:18.074171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1082] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:18.079570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.113743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.126420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c707f5-da25-471f-9311-d23d86794215 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.136432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.165671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.166504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.166784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.167068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.177277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:18.181097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca2cb-f97e-44c8-826f-b94d5f61a2e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:18.185247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:18.186372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:18.187311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1083] 10.222.0.16 () {56 vars in 1120 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:18.192574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.215208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.215887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.216170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.216495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.217247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:18.217553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.218933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.227717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1084] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:18.233472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.294321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8efad-bafb-4ce0-ba17-03d8c96f202d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.327100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.330615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.331288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.331524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473f8a-3d86-4875-b870-b2343ea93e2a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.331675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.331898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.332635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:18.332945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.335027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.342422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1085] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:18.348200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.367795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.368728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.369018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.369302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] PATH_INFO: `/v3/users/ca485756c8ed4d3580d3269c865c71ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.370532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] RBAC: Authorizing `identity:get_user(user_id=ca485756c8ed4d3580d3269c865c71ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.371985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.373691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1086] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/users/ca485756c8ed4d3580d3269c865c71ae => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:18.378067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.380075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.447718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c685c-6522-4316-a107-4aa6a417f13c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.479700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf3b2b-939a-46f9-9508-7e06e81a3806 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.481218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.482028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.482313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.482590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.493676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.495350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:18.512626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:18.513620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:18.514690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:18.514690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:18.514690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:18.514690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:18.515241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.515241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.515241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.515241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.515512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1087] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:18.515639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.517108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.519380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1088] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:18.519800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.524859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.559894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.600748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.618841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848c88c-98dc-4c3b-87f2-ff7e5cb8e1a5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.627772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28a762-ff9a-454b-9564-a9e0173f9474 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.652647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.653467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.653757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.654038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.654764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:18.655155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.656639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.661963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.662866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.663153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.663443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632/users/ca485756c8ed4d3580d3269c865c71ae/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.665494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] RBAC: Authorizing `identity:create_grant(project_id=e4d6900a26954f29afed8163cffc4632, user_id=ca485756c8ed4d3580d3269c865c71ae, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.666942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.668488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1089] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:18.674133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.680988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1090] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:57:18 2022] PUT /identity/v3/projects/e4d6900a26954f29afed8163cffc4632/users/ca485756c8ed4d3580d3269c865c71ae/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 02:57:18.790707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93068b1a-51d2-4804-b010-afef338ced7b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:18.829302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:18.830488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:18.830488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:18.830676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:18.838754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:18.840323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:57:18.855938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:18.857054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:18.858539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1091] 10.222.0.16 () {56 vars in 1093 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:57:18.863280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:18.961423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:18.969282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6bd81-d2ae-48f8-aee1-e95637eceb6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:19.005738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:19.006404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:19.006689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:19.006959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:19.007780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:19.008142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:19.009616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:19.027946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1092] 10.222.0.16 () {56 vars in 1133 bytes} [Tue Jun 21 02:57:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:19.034247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:19.037325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:19.140658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158e2a7-35c9-49dc-8ddd-bbb9d525f57d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:19.176091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:19.176788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:19.177074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:19.177350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:19.245704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:19.247186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:19.249154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:19.349422 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:19.355465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:19.456372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce0c0e-b62f-4878-8409-c2c8e8be934f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:19.490935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:19.491718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:19.491997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:19.492269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:19.502962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:19.504444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:19.522094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:19.523103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:19.524021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:57:19.524021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:19.524021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:57:19.524021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:19.528074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1094] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:19.529062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:19.631257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb919-dfb2-4558-b509-2cb4a81bccd8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:19.632837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:19.669720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:19.670387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:19.670667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:19.670939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:19.671665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:19.671972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:19.673396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:19.685428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1095] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:19.690997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:19.695051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:19.796253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54861b16-adea-4edf-8444-8c0f0f57f91e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:19.833799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:19.834606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:19.834892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:19.835169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:19.842176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:19.843638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:57:19.857718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:19.858660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:19.859555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1096] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:57:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:19.865060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:19.968157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b12ea-d35f-46e9-b28a-c5854e6c7de5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:20.003173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:20.003836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:20.004127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:20.004431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:20.005231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:20.005548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:20.007012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26392f7c-a450-405c-832b-f4f7d471c62f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:20.019189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1097] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:57:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:20.025348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:20.086908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:20.125296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01591d-871c-40f5-a9ac-4c1b0d9df50c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:20.162581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:20.163534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:20.163783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:20.164093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:20.164945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] RBAC: Adding query filter params (role.id=726dc0d8fe874f0f9c4201e2d3091c9b, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=1c16671fd5844a77ac6e56dff36f86f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:20.165257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:20.166702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353841c0-0491-4bdc-8889-ea50f73ed88d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:20.173358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:20 2022] GET /identity/v3/role_assignments?user.id=1c16671fd5844a77ac6e56dff36f86f8&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=726dc0d8fe874f0f9c4201e2d3091c9b => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:20.754777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d30-dfe2-4c57-ba46-9af6de1ab834 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:20.755033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d30-dfe2-4c57-ba46-9af6de1ab834 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:20.755329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd07d30-dfe2-4c57-ba46-9af6de1ab834 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:20.756878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1099] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:20.762696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:20.762988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:20.763262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:20.781597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:20.782848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:20.853400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1100] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:20.877632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07d30-dfe2-4c57-ba46-9af6de1ab834 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:20.903165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:20.984626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd07d30-dfe2-4c57-ba46-9af6de1ab834 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:21.019263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:21.019996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:21.020282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:21.020591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:21.030827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:21.038848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:21.039937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:21.041317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:21.046393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:21.110326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:21.161369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ffec-e13a-45a6-b293-90b3ec53b450 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:21.197084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:21.197748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:21.198030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:21.198301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:21.199042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:21.199355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:21.200752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:21.209466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:21.215571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:21.313443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:21.339783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55a338-9fdf-42de-bbb6-5b66683b5379 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:21.378046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:21.379011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:21.379298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:21.379589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] PATH_INFO: `/v3/users/ca485756c8ed4d3580d3269c865c71ae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:21.380953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] RBAC: Authorizing `identity:get_user(user_id=ca485756c8ed4d3580d3269c865c71ae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:21.382410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:21.384082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:21 2022] GET /identity/v3/users/ca485756c8ed4d3580d3269c865c71ae => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:21.390372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:21.499537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0024cf5-90e1-4ec0-badc-a196f21b4436 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:21.519911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:21.539486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:21.540135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:21.540444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:21.540720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:21.612666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:21.614039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:21.721991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:21.729058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:21.848719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7820fe-9f6b-4f04-9dfc-dc6a1f7b4856 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:21.885039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:21.885794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:21.886106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:21.886372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:21.887680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:21.889165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a928-2530-4050-baeb-901b1745c1e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:21.890863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:21 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:21.900274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:21.924381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:22.008439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2e67f-2fa3-4dfd-a3be-5af7e9a6880f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:22.048400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:22.049213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:22.049509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:22.049789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:22.050629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=e4d6900a26954f29afed8163cffc4632, user.id=ca485756c8ed4d3580d3269c865c71ae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:22.050946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:22.052389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519f8aa-ef63-4887-8ac1-c09df56e99fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:22.059026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:21 2022] GET /identity/v3/role_assignments?user.id=ca485756c8ed4d3580d3269c865c71ae&scope.project.id=e4d6900a26954f29afed8163cffc4632&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:24.085986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce83ba-4979-4184-a6ca-2eb55a5d749d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:24.086909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce83ba-4979-4184-a6ca-2eb55a5d749d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:24.086909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce83ba-4979-4184-a6ca-2eb55a5d749d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:24.089150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1107] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:24.095736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:24.096444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:24.096444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:24.119924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:24.121167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:24.192114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:24.219554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce83ba-4979-4184-a6ca-2eb55a5d749d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:24.243286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:24.329090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce83ba-4979-4184-a6ca-2eb55a5d749d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:24.367945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:24.368768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:24.369052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:24.369327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:24.370574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:24.372034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2508959-a1b0-4898-bdce-0928c93cf075 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:24.373696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:24.383151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:24.402223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:24.498725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18aa3-5720-42e7-ac44-48347f51dcdd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:24.536129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:24.536835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:24.537111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:24.537387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:24.538107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:24.539540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce770a-78ab-4e5d-94fe-c859c0595a38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:24.563158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:57:24 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:26.608073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f36a8-8dce-4a1e-9a41-6997fa083c5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:26.608705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f36a8-8dce-4a1e-9a41-6997fa083c5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:26.608705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f36a8-8dce-4a1e-9a41-6997fa083c5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:26.610052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1111] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:26.616477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:26.617214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:26.617363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:26.644776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:26.645982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:26.713489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:26.719216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:26.757526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f36a8-8dce-4a1e-9a41-6997fa083c5d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:26.860476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:26.877958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f36a8-8dce-4a1e-9a41-6997fa083c5d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:26.917301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:26.918163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:26.918442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:26.918719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] PATH_INFO: `/v3/roles/692ec53a198846cda9892422cde67a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:26.920245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] RBAC: Authorizing `identity:get_role(role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:26.921685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:26.923070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:26 2022] GET /identity/v3/roles/692ec53a198846cda9892422cde67a24 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:26.932571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:27.039497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1987e41-8f06-418b-9e19-c828d987fb43 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:27.074749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:27.075619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:27.075901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:27.076175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] PATH_INFO: `/v3/users/39c080fa6b2640d2abcee322d58e042b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:27.090212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] RBAC: Authorizing `identity:get_user(user_id=39c080fa6b2640d2abcee322d58e042b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:27.091664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:27.093352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:26 2022] GET /identity/v3/users/39c080fa6b2640d2abcee322d58e042b => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:27.099395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:27.234906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce1aa3-5839-4c7f-9787-59640e66f11e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:27.271875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:27.272560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:27.272847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:27.273126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:27.297832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:27.350668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:27.352145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:27.456316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:27.463092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:27.529028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:27.574102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaddc12-0803-4bb2-b2f8-cf2ec6e2dc2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:27.619838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111e907-f304-430e-b272-1957488ead46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:27.621212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e907-f304-430e-b272-1957488ead46 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:27.621673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e907-f304-430e-b272-1957488ead46 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:27.622123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111e907-f304-430e-b272-1957488ead46 None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:27.624186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111e907-f304-430e-b272-1957488ead46 None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:27.627034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111e907-f304-430e-b272-1957488ead46 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:27.629001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:27 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:27.638563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:27.710519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:27.761133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4c7dd-a15a-4495-b616-8dcb0a84d83e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:27.800537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:27.801182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:27.801464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:27.801733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:27.802566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=e4d6900a26954f29afed8163cffc4632, user.id=39c080fa6b2640d2abcee322d58e042b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:27.802878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:27.804316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be207b9-0c7b-484d-82ad-5688ba4fe313 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:27.810408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:27 2022] GET /identity/v3/role_assignments?user.id=39c080fa6b2640d2abcee322d58e042b&scope.project.id=e4d6900a26954f29afed8163cffc4632&role.id=692ec53a198846cda9892422cde67a24 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:29.785335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e6a9-3aa5-47cf-b998-aa1095907ca6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:29.786257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e6a9-3aa5-47cf-b998-aa1095907ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:29.786257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e6a9-3aa5-47cf-b998-aa1095907ca6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:29.788594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1118] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:29.794997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:29.795427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:29.795567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:29.821102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:29.822294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:29.892715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:29.921389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e6a9-3aa5-47cf-b998-aa1095907ca6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:30.040788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e6a9-3aa5-47cf-b998-aa1095907ca6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:30.066706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:30.081667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:30.082531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:30.082812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:30.083083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] PATH_INFO: `/v3/roles/692ec53a198846cda9892422cde67a24` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:30.084747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] RBAC: Authorizing `identity:get_role(role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:30.086136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:30.087506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:29 2022] GET /identity/v3/roles/692ec53a198846cda9892422cde67a24 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:30.093804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:30.144426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:30.207224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab38e2b-f9b9-4e8c-adbc-b16c7f6b7c45 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:30.243295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:30.244167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:30.244456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:30.244760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] PATH_INFO: `/v3/users/39c080fa6b2640d2abcee322d58e042b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:30.246160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] RBAC: Authorizing `identity:get_user(user_id=39c080fa6b2640d2abcee322d58e042b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:30.247621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:30.249302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:30 2022] GET /identity/v3/users/39c080fa6b2640d2abcee322d58e042b => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:30.260754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:30.380655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34839c1d-1354-45d1-b0ed-a1fcdb798bcf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:30.415555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:30.416451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:30.416718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:30.416991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:30.418285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:30.419713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeea3bf-ef19-475d-a181-f0a5656d0ecf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:30.421454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:30 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:30.427662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:30.539099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f898-47b7-41e0-b2f3-32d5e978750c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:30.558425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:30.577250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:30.578147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:30.578427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:30.578700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632/users/39c080fa6b2640d2abcee322d58e042b/roles/692ec53a198846cda9892422cde67a24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:30.580922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4d6900a26954f29afed8163cffc4632, user_id=39c080fa6b2640d2abcee322d58e042b, role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:30.582375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380b368-2a49-41fd-8457-e923b3f81747 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:30.598200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:57:30 2022] PUT /identity/v3/projects/e4d6900a26954f29afed8163cffc4632/users/39c080fa6b2640d2abcee322d58e042b/roles/692ec53a198846cda9892422cde67a24 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:57:32.592838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1545e-8a0a-433b-92d5-a2b58107697b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:32.593751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1545e-8a0a-433b-92d5-a2b58107697b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:32.593751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1545e-8a0a-433b-92d5-a2b58107697b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:32.595995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1124] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:32.601561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad629696-8c69-496e-91d4-211608839222 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:32.602001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad629696-8c69-496e-91d4-211608839222 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:32.602127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad629696-8c69-496e-91d4-211608839222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:32.619902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad629696-8c69-496e-91d4-211608839222 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:32.621138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad629696-8c69-496e-91d4-211608839222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:32.692850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:32.717176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1545e-8a0a-433b-92d5-a2b58107697b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:32.788001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:32.830930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1545e-8a0a-433b-92d5-a2b58107697b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:32.868600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:32.869382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:32.869667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:32.869936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] PATH_INFO: `/v3/roles/692ec53a198846cda9892422cde67a24` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:32.871444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] RBAC: Authorizing `identity:get_role(role_id=692ec53a198846cda9892422cde67a24)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:32.872860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:32.874239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:32 2022] GET /identity/v3/roles/692ec53a198846cda9892422cde67a24 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:32.882264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad629696-8c69-496e-91d4-211608839222 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:32.983886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad629696-8c69-496e-91d4-211608839222 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:32.991889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:33.021038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:33.021913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:33.022187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:33.022450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] PATH_INFO: `/v3/users/39c080fa6b2640d2abcee322d58e042b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:33.023683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] RBAC: Authorizing `identity:get_user(user_id=39c080fa6b2640d2abcee322d58e042b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:33.025186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:33.026806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:32 2022] GET /identity/v3/users/39c080fa6b2640d2abcee322d58e042b => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:33.032942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:33.139621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fce60-8e78-4f39-9b93-8983f1f13907 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:33.172114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:33.179803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:33.180465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:33.180782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:33.181066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:33.252510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:33.254206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:33.356186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:33.363012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:33.475202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e58c9a-e547-4879-b1a2-b29f2fdfd859 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:33.511302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:33.512183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:33.512490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:33.512767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] PATH_INFO: `/v3/projects/e4d6900a26954f29afed8163cffc4632` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:33.513966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] RBAC: Authorizing `identity:get_project(project_id=e4d6900a26954f29afed8163cffc4632)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:33.515439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697278-b946-4e47-9b1a-7a7145172cbc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:33.517215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:33 2022] GET /identity/v3/projects/e4d6900a26954f29afed8163cffc4632 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:33.523650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:33.553624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:33.621671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed69a-ca3c-41c2-bfff-90d78d7ccd86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:33.656647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:33.657306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:33.657590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:33.657864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:33.658692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] RBAC: Adding query filter params (role.id=692ec53a198846cda9892422cde67a24, scope.project.id=e4d6900a26954f29afed8163cffc4632, user.id=39c080fa6b2640d2abcee322d58e042b) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:33.659001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:33.660763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c884-267e-495a-b35b-0f93627312ca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:33.666930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:33 2022] GET /identity/v3/role_assignments?user.id=39c080fa6b2640d2abcee322d58e042b&scope.project.id=e4d6900a26954f29afed8163cffc4632&role.id=692ec53a198846cda9892422cde67a24 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:35.670320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db7b1-32bc-4738-89cc-e7a01f8f4bd8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:35.671227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db7b1-32bc-4738-89cc-e7a01f8f4bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:35.671227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26db7b1-32bc-4738-89cc-e7a01f8f4bd8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:35.673452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1131] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:35.679807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:35.680516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:35.680698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:35.701280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:35.702473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:35.772288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:35.796112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db7b1-32bc-4738-89cc-e7a01f8f4bd8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:35.848747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:35.909026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26db7b1-32bc-4738-89cc-e7a01f8f4bd8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:35.946941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:35.947668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:35.947951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:35.948228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:35.949547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:35.950983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876eef76-1382-4ac0-8932-3634f942325f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:35.952662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:35.959619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:35.994957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:36.073569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b5af7-aab0-4bff-afe4-a12ffeacddff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:36.108961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:36.109645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:36.109930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:36.110207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:36.110919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:36.112380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b70de-87f4-4814-bea1-85839c84b427 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:36.146566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:57:35 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:38.195027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2875ac-d5c7-45e6-873b-6918d75d3e34 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:38.195774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2875ac-d5c7-45e6-873b-6918d75d3e34 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:38.195865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2875ac-d5c7-45e6-873b-6918d75d3e34 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:38.198060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1135] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:38.203304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:38.203735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:38.203847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:38.222315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:38.223505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:38.302413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:38.326860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2875ac-d5c7-45e6-873b-6918d75d3e34 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:38.452975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2875ac-d5c7-45e6-873b-6918d75d3e34 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:38.464614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:38.500152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:38.501333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:38.501616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:38.501889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:38.503238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:38.504768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c67255-dea3-4e37-9ba0-fa12e043d0ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:38.506434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:38.514947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:38.524007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:38.626355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c005098-1502-4071-acb4-f9de88d9d96b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:38.662438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:38.663100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:38.663385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:38.663659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:38.664371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:38.665849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83ddfd-8986-43d5-918a-864eefceba79 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:38.690587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:57:38 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:40.742798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0524-4a73-4658-9b35-9d9965fef647 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:40.743395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0524-4a73-4658-9b35-9d9965fef647 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:40.743395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0524-4a73-4658-9b35-9d9965fef647 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:40.744781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1139] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:40.751535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:40.752312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:40.752518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:40.776955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:40.778169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:40.842255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:40.854611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:40.879482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0524-4a73-4658-9b35-9d9965fef647 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:41.009181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0524-4a73-4658-9b35-9d9965fef647 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:41.044521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:41.045308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:41.045582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:41.045853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:41.061101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:41.070735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:41.071950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:41.073014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:41.079804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:41.184989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74e62b-9e84-42f3-b694-1765c579a4a7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:41.220689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:41.221358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:41.221619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:41.221911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:41.222708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:41.223019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:41.224404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:41.232832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:41.239240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:41.292362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:41.355660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fbb5c-21fc-4f37-9904-2c67d1e48b3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:41.392016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:41.392930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:41.393216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:41.393492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] PATH_INFO: `/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:41.407668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] RBAC: Authorizing `identity:get_user(user_id=9fb9fb0bb64b4dd7867ad01df2cf7959)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:41.409178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:41.410898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:41 2022] GET /identity/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:41.419684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:41.480196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:41.527804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a011-67f6-4180-b225-d1977480de6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:41.563778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:41.564356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:41.564681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:41.564938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:41.646738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:41.648243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:41.707389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:41.759203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:41.765879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:41.863642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:41.871021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a367eaf-fc4d-49ec-931d-0b2b9b0765ce None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:41.907249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:41.908132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:41.908442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:41.908714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] PATH_INFO: `/v3/projects/678468702f3b4fddb194603ec89ef5c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:41.909998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] RBAC: Authorizing `identity:get_project(project_id=678468702f3b4fddb194603ec89ef5c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:41.911442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3c170-7a8f-456e-bb5e-a6504bc08488 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:41.913195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:41 2022] GET /identity/v3/projects/678468702f3b4fddb194603ec89ef5c2 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:41.919515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:42.027804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc023254-4210-4dcc-919e-c1b51fc0bf99 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:42.064529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:42.065173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:42.065449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:42.065761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:42.066579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=678468702f3b4fddb194603ec89ef5c2, user.id=9fb9fb0bb64b4dd7867ad01df2cf7959) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:42.066889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:42.068312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a7c75-fe51-498b-9128-b069c46da018 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:42.073835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:41 2022] GET /identity/v3/role_assignments?user.id=9fb9fb0bb64b4dd7867ad01df2cf7959&scope.project.id=678468702f3b4fddb194603ec89ef5c2&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:44.078093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e5eb4-4984-4ad9-9b44-4a69494353b1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.078996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e5eb4-4984-4ad9-9b44-4a69494353b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.078996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e5eb4-4984-4ad9-9b44-4a69494353b1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.081170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1147] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:44.086720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.087149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.087239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.105891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:44.107065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:44.121884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:44.179445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:44.202609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e5eb4-4984-4ad9-9b44-4a69494353b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:44.313269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e5eb4-4984-4ad9-9b44-4a69494353b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:44.349252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:44.349977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.350260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.350534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.366430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:44.374018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:44.375134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:44.376364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:44.382529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:44.488473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fabe3-34f7-4ee2-b7d6-3cc28a6409ac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:44.523464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:44.524122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.524445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.524697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.525450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:44.525755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:44.527137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:44.535261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:44.543943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:44.572677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:44.655538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4ad25-e84b-4447-b26a-fe9eaef07ab7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:44.691312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:44.692186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.692491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.692767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] PATH_INFO: `/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.694070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb9fb0bb64b4dd7867ad01df2cf7959)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:44.695518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:44.697215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:44 2022] GET /identity/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:44.703987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:44.724835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:44.818847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9ebf1-14a2-4731-94c0-027c81d86508 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:44.853764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:44.854629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:44.854913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:44.855195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] PATH_INFO: `/v3/projects/678468702f3b4fddb194603ec89ef5c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:44.856459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] RBAC: Authorizing `identity:get_project(project_id=678468702f3b4fddb194603ec89ef5c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:44.857899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610f27a-4aaf-4e98-9d41-900502ad99b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:44.859628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:44 2022] GET /identity/v3/projects/678468702f3b4fddb194603ec89ef5c2 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:44.866106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:44.970895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:44.981102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25f9fc-cf35-46e8-bf43-6617c55e5b81 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:45.016245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:45.017174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:45.017453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:45.017723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] PATH_INFO: `/v3/projects/678468702f3b4fddb194603ec89ef5c2/users/9fb9fb0bb64b4dd7867ad01df2cf7959/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:45.019950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] RBAC: Authorizing `identity:create_grant(project_id=678468702f3b4fddb194603ec89ef5c2, user_id=9fb9fb0bb64b4dd7867ad01df2cf7959, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:45.021429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb494c-0bee-4220-a346-6b1a1539df30 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:45.039494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:57:44 2022] PUT /identity/v3/projects/678468702f3b4fddb194603ec89ef5c2/users/9fb9fb0bb64b4dd7867ad01df2cf7959/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:57:47.068281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e53c-d082-49a0-9a34-7caed5956533 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.069470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e53c-d082-49a0-9a34-7caed5956533 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.069948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e53c-d082-49a0-9a34-7caed5956533 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.072463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1154] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:47.081282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.081969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.082159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.105848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:47.107049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:47.180041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:47.209031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e53c-d082-49a0-9a34-7caed5956533 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:47.299129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:47.324839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e53c-d082-49a0-9a34-7caed5956533 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:47.361101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:47.361895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.362182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.362663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.373883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:57:47.381973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:47.383053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:57:47.383977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:47.389552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:47.479421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:47.502563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d7956-5e4a-4386-8064-976454b73d5e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:47.539879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:47.540573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.540854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.541130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.541880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:47.542194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:47.543575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:47.552450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:47.561974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:47.687572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f806-8265-415a-8262-cd14365d621d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:47.724173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:47.725085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.725363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.725637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] PATH_INFO: `/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.726942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] RBAC: Authorizing `identity:get_user(user_id=9fb9fb0bb64b4dd7867ad01df2cf7959)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:47.728383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:47.730035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:57:47 2022] GET /identity/v3/users/9fb9fb0bb64b4dd7867ad01df2cf7959 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:47.736237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:47.842588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dca203-3073-4825-bc96-19fd747af574 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:47.878551 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:47.879190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:47.879471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:47.879752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:47.895461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:47.953680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:47.955082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:48.054930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:48.061148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:48.071244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:48.165648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755da519-4f68-48ff-a249-8e07dc815094 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:48.201392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:48.202242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:48.202521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:48.202795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] PATH_INFO: `/v3/projects/678468702f3b4fddb194603ec89ef5c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:48.204079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] RBAC: Authorizing `identity:get_project(project_id=678468702f3b4fddb194603ec89ef5c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:48.205487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7eeb2-cceb-4b79-ae50-7f03759601d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:48.207385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:57:48 2022] GET /identity/v3/projects/678468702f3b4fddb194603ec89ef5c2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:48.214332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:48.270900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:48.333550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6d796-c5a0-409d-ab2f-c98305f201ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:48.367379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:48.368022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:48.368304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:48.368616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:48.369447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=678468702f3b4fddb194603ec89ef5c2, user.id=9fb9fb0bb64b4dd7867ad01df2cf7959) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:48.369761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:48.371211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87242c8a-0bbe-4b8a-a2d4-931daaf37098 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:48.377473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:57:48 2022] GET /identity/v3/role_assignments?user.id=9fb9fb0bb64b4dd7867ad01df2cf7959&scope.project.id=678468702f3b4fddb194603ec89ef5c2&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:50.392961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb0508-eabe-41c1-a07f-8e60950a7c99 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:50.393926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb0508-eabe-41c1-a07f-8e60950a7c99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:50.393926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb0508-eabe-41c1-a07f-8e60950a7c99 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:50.396300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1162] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:50.405162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:50.405862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:50.406010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:50.431477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:50.432644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:50.517125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:50.540244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb0508-eabe-41c1-a07f-8e60950a7c99 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:50.630076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:50.657898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb0508-eabe-41c1-a07f-8e60950a7c99 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:50.693390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:50.693965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:50.694246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:50.694522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:50.770706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:50.772180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:50.849152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:50.876248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:57:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:57:50.883329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:50.931355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:50.998394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3edde-4dd6-43a6-8517-7d69ae589cda None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:51.034863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:51.035661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:51.035939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:51.036211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:51.043531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:51.045020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7e615871-f1e9-4992-b541-586a00c550ee None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:57:51.060042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:57:51.061254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:57:51.062663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:57:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:57:51.070717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:51.180381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7edd18-a8bf-405a-8d2e-ae894d9e8fb7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:51.215167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:51.215822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:51.216102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:51.216375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:51.217134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:57:51.217447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:51.218820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619224-1422-49d0-9b9c-d1360187d372 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:51.226120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Tue Jun 21 02:57:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:57:53.203718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8675-dfc5-4005-adec-5dda3776ce06 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:53.204244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8675-dfc5-4005-adec-5dda3776ce06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:53.204244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c8675-dfc5-4005-adec-5dda3776ce06 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:53.206474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1167] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:53.212990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551438c1-42f4-4321-bf0f-70a3353b3b76 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:53.213713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551438c1-42f4-4321-bf0f-70a3353b3b76 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:53.213874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551438c1-42f4-4321-bf0f-70a3353b3b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:53.241523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-551438c1-42f4-4321-bf0f-70a3353b3b76 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:53.242742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-551438c1-42f4-4321-bf0f-70a3353b3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:53.276193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:53.315856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:53.340710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c8675-dfc5-4005-adec-5dda3776ce06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:53.467955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:53.475739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c8675-dfc5-4005-adec-5dda3776ce06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:53.516066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:53.516762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:53.517046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:53.517313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:53.518051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:53.519642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184ab21-bee4-42ea-a4d7-22a78fe727e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:53.540745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Tue Jun 21 02:57:53 2022] POST /identity/v3/domains => generated 267 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:55.578683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217dac5-b9b9-4a52-8122-740345752222 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:55.579315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217dac5-b9b9-4a52-8122-740345752222 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:55.579315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217dac5-b9b9-4a52-8122-740345752222 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:55.580774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1170] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:55.587255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:55.588059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:55.588186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:55.616450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:55.617660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:55.695236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:57:55.719574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217dac5-b9b9-4a52-8122-740345752222 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:55.830468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217dac5-b9b9-4a52-8122-740345752222 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:55.865092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:55.865948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:55.866233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:55.866506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] PATH_INFO: `/v3/domains/d0b63fbd3181467295148807b89d1ae0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:55.877640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d0b63fbd3181467295148807b89d1ae0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:55.879181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e6223-8e25-449f-a0af-b6920e12425c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:55.880858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Jun 21 02:57:55 2022] GET /identity/v3/domains/d0b63fbd3181467295148807b89d1ae0 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:55.887397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:55.992876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:56.006694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880cca3-f97d-4d26-8cee-a9c566ef23eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:56.040291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:56.040984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:56.041276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:56.041551 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:56.042268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:56.043721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc687a4d-8108-466b-a6b8-2a0355363300 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:56.071446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:57:55 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:57:58.118811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702a70-56b4-40ad-bc10-8f6083ebcfef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:58.119401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702a70-56b4-40ad-bc10-8f6083ebcfef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:58.119401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04702a70-56b4-40ad-bc10-8f6083ebcfef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:58.121591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1174] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:57:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:57:58.130412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:58.130907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:58.130907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:58.148610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:57:58.149810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:57:58.229783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:57:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 02:57:58.264943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04702a70-56b4-40ad-bc10-8f6083ebcfef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:58.295895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:58.375407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04702a70-56b4-40ad-bc10-8f6083ebcfef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:58.412364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:58.413273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:58.413551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:58.413826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] PATH_INFO: `/v3/domains/d0b63fbd3181467295148807b89d1ae0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:58.415150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d0b63fbd3181467295148807b89d1ae0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:58.416636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae2aa1-c9a3-484b-8222-91aefb382cf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:58.418269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Tue Jun 21 02:57:58 2022] GET /identity/v3/domains/d0b63fbd3181467295148807b89d1ae0 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:57:58.424989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:57:58.506585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:57:58.539993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8ef3-7a1d-4d07-89e8-42fa066c0c93 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:57:58.575377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:57:58.576082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:57:58.576363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:57:58.576677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:57:58.577389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:57:58.578839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a194b-ad6b-4030-845d-20212d747aad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:57:58.602345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:57:58 2022] POST /identity/v3/users => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:58:00.645861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddab8d4-ca76-4c37-8543-fddce0392e58 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:00.646462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddab8d4-ca76-4c37-8543-fddce0392e58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:00.646462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddab8d4-ca76-4c37-8543-fddce0392e58 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:00.648014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1178] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:58:00.654641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:00.655373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:00.655497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:00.682550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:58:00.683735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:58:00.756438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:58:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:58:00.780986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddab8d4-ca76-4c37-8543-fddce0392e58 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:00.886555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:00.898567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddab8d4-ca76-4c37-8543-fddce0392e58 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:00.932686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:00.933478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:00.933764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:00.934038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:00.945005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:58:00.952587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:58:00.953751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:58:00.955011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:58:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:58:00.960235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:01.023060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:01.074365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b6d19-9549-4a3b-88e6-a7bd4a666682 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:01.114812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:01.115477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:01.115760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:01.116225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:01.117116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:58:01.117427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:01.118834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:01.128277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:58:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:01.135698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:01.246816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b294c-f535-4bca-a1d4-6f1f245148b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:01.282891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:01.283765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:01.284050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:01.284325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] PATH_INFO: `/v3/users/69b8b7a184594c2e953ccebab8d93d4e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:01.297981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] RBAC: Authorizing `identity:get_user(user_id=69b8b7a184594c2e953ccebab8d93d4e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:01.299430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:01.301114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:58:01 2022] GET /identity/v3/users/69b8b7a184594c2e953ccebab8d93d4e => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:01.310159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:01.413470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734b790-a4b3-4b58-a0cd-95133bde5013 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:01.430586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:01.450287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:01.450936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:01.451217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:01.451490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:01.521863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:01.523497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:01.623706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:58:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:58:01.630291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:01.648418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:01.735044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd2196-6b23-4454-9b3a-8083334ed200 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:01.773008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:01.773860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:01.774144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:01.774417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] PATH_INFO: `/v3/projects/63feeb3226354dd891547904790a4a28` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:01.775716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] RBAC: Authorizing `identity:get_project(project_id=63feeb3226354dd891547904790a4a28)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:01.777191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434faba-3290-45e9-a131-521733b51c3d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:01.778897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:58:01 2022] GET /identity/v3/projects/63feeb3226354dd891547904790a4a28 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:01.785486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:01.804356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:01.888726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d2640-48f6-4fb2-8fc5-9bdecea37744 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:01.924739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:01.925406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:01.925689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:01.925963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:01.926803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=63feeb3226354dd891547904790a4a28, user.id=69b8b7a184594c2e953ccebab8d93d4e) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:58:01.927113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:01.928637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578a52-fff8-4372-9b6b-5357c49a0e4d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:01.934136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:58:01 2022] GET /identity/v3/role_assignments?user.id=69b8b7a184594c2e953ccebab8d93d4e&scope.project.id=63feeb3226354dd891547904790a4a28&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:03.900227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d12a2-ae8d-4859-ac0e-f0d44b86fcd1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:03.901233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d12a2-ae8d-4859-ac0e-f0d44b86fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:03.901233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d12a2-ae8d-4859-ac0e-f0d44b86fcd1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:03.903346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1186] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:58:03.908791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:03.909233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:03.909340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:03.927914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:58:03.929159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:58:03.997680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:58:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:58:04.020894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d12a2-ae8d-4859-ac0e-f0d44b86fcd1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:04.112284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:04.140858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d12a2-ae8d-4859-ac0e-f0d44b86fcd1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:04.176212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:04.177117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:04.177356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:04.177657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:04.188656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:58:04.196976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:58:04.198000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:58:04.199044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:58:04.204559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:04.219426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:04.308737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885b267-3f2f-4b34-8d60-f2b9fa5c6854 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:04.344128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:04.344819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:04.345108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:04.345385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:04.346132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:58:04.346445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:04.347833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:04.356432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:04.365568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:04.480806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb66b8d-7afa-4d9a-bfe1-1ee875d07014 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:04.530112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:04.531405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:04.531780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:04.532152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] PATH_INFO: `/v3/users/69b8b7a184594c2e953ccebab8d93d4e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:04.534278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] RBAC: Authorizing `identity:get_user(user_id=69b8b7a184594c2e953ccebab8d93d4e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:04.536498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:04.539071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:58:04 2022] GET /identity/v3/users/69b8b7a184594c2e953ccebab8d93d4e => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:04.547943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:04.630566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:04.660727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87da0b-d84c-490b-826f-487e1301e25f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:04.701094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:04.702296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:04.702635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:04.702969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] PATH_INFO: `/v3/projects/63feeb3226354dd891547904790a4a28` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:04.705010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] RBAC: Authorizing `identity:get_project(project_id=63feeb3226354dd891547904790a4a28)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:04.707243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa248-6db7-4015-919c-b26d99c3011e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:04.709909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:58:04 2022] GET /identity/v3/projects/63feeb3226354dd891547904790a4a28 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:04.716664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:04.737369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:04.833004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be890d-aca9-4270-bde0-428741d694a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:04.868136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:04.869056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:04.869336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:04.869612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] PATH_INFO: `/v3/projects/63feeb3226354dd891547904790a4a28/users/69b8b7a184594c2e953ccebab8d93d4e/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:04.871832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] RBAC: Authorizing `identity:create_grant(project_id=63feeb3226354dd891547904790a4a28, user_id=69b8b7a184594c2e953ccebab8d93d4e, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:04.873302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02d88-db4e-4b7a-82b2-22aed16224be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:04.890027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:58:04 2022] PUT /identity/v3/projects/63feeb3226354dd891547904790a4a28/users/69b8b7a184594c2e953ccebab8d93d4e/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:58:06.887611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e01e7-b1f6-400d-b971-d584b1b02fed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:06.888493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e01e7-b1f6-400d-b971-d584b1b02fed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:06.888493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e01e7-b1f6-400d-b971-d584b1b02fed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:06.890822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1193] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:58:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:58:06.897379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:06.898019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:06.898019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:06.915040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:58:06.916224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:58:06.979407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:06.985022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:58:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:58:07.009448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e01e7-b1f6-400d-b971-d584b1b02fed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:07.125244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e01e7-b1f6-400d-b971-d584b1b02fed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:07.160701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:07.161426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:07.161709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:07.161989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:07.173146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:58:07.180807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:58:07.181886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:58:07.190663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:58:07.191501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:07.294517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8453-2b43-4ae6-9d3b-d88b10ed4f23 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:07.329884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:07.330542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:07.330822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:07.331098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:07.331844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:58:07.332155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:07.333583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:07.341888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:07.348753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:07.352577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:07.469032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2264c9-1c4f-4197-900e-78d1e4a26e49 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:07.514319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:07.515178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:07.515460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:07.515728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] PATH_INFO: `/v3/users/69b8b7a184594c2e953ccebab8d93d4e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:07.516993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] RBAC: Authorizing `identity:get_user(user_id=69b8b7a184594c2e953ccebab8d93d4e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:07.518433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:07.520056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Tue Jun 21 02:58:07 2022] GET /identity/v3/users/69b8b7a184594c2e953ccebab8d93d4e => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:07.526843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:07.607252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:07.651387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43479-bba5-499a-b4fd-8ccd0a9197a6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:07.692943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:07.693966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:07.694670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:07.695130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:07.773817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:07.775204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:07.837178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:07.878440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:58:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:58:07.887583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:07.964724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:07.997026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450be5d-774f-4b88-9996-abfc6a07d5ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:08.047482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:08.048755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:08.049168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:08.049596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] PATH_INFO: `/v3/projects/63feeb3226354dd891547904790a4a28` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:08.051705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] RBAC: Authorizing `identity:get_project(project_id=63feeb3226354dd891547904790a4a28)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:08.054018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4ed8c-326d-4098-a0ef-1b9c84d1ab5d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:08.056750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Tue Jun 21 02:58:07 2022] GET /identity/v3/projects/63feeb3226354dd891547904790a4a28 => generated 361 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:08.063641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:58:08.171157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922389c-a84e-4670-abf2-d652be97112e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:58:08.208814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:58:08.209470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:58:08.209764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:58:08.210066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:58:08.210894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=63feeb3226354dd891547904790a4a28, user.id=69b8b7a184594c2e953ccebab8d93d4e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:58:08.211210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:58:08.212673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1344-4215-4c3e-9041-161338106d95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:58:08.218761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:58:08 2022] GET /identity/v3/role_assignments?user.id=69b8b7a184594c2e953ccebab8d93d4e&scope.project.id=63feeb3226354dd891547904790a4a28&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:58:18.038352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=31 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:58:18.215964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:03.585159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c408-42cd-4310-b2d0-2aec342a5438 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:03.586718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c408-42cd-4310-b2d0-2aec342a5438 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:03.587170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c408-42cd-4310-b2d0-2aec342a5438 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:03.589307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1201] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:03.595638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:03.596131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:03.596237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:03.615514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:03.617011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:03.693474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:03.721215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c408-42cd-4310-b2d0-2aec342a5438 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:03.852608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c408-42cd-4310-b2d0-2aec342a5438 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:03.893644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:03.894441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:03.894738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:03.894997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:03.909042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:03.910932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:03.933317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:03.934405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:03.935984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:03.935984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:03.935984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:03.935984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:03.935984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:03.941263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:04.092342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da762283-29e5-4cd3-a9ae-1d0b3e43d17b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:04.139889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:04.154859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:04.155676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:04.156257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:04.156689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:04.157424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:04.157732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:04.159127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dcb5c-fea4-457e-826e-84df834b49fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:04.172139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:04.178598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:04.285001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:04.339933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7e111-d5ae-4f70-b04b-14d071bebb0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:04.382267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:04.382946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:04.383226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:04.383506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:04.384210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:04.385738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cb06a-c7de-4229-8ff9-4cda8b52e976 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:04.409351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Tue Jun 21 02:59:04 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:59:06.511060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d8f8-ad36-4803-84e4-ffc315aa9a9b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:06.511060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d8f8-ad36-4803-84e4-ffc315aa9a9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:06.511060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6d8f8-ad36-4803-84e4-ffc315aa9a9b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:06.511060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1206] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 02:59:06.516443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:06.516862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:06.517098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:06.534501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:06.535683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:06.619262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:06.644602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6d8f8-ad36-4803-84e4-ffc315aa9a9b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:06.766415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6d8f8-ad36-4803-84e4-ffc315aa9a9b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:06.808064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:06.809972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:06.810357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:06.810692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:06.824653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:06.833257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:06.834405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:06.835309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:06.839236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:06.841281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:06.885762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:06.964361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be9b84-1b3e-439f-9aa2-374674868b9d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.007579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.008327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.008766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.009072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.009865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:07.010260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:07.011813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:07.025907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:07.027160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:07.084339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:07.144736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba517a-34a1-4f10-bd6b-ba943c2784e8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.187491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.188733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.189046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.189317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.202083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:07.203715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:07.229352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:07.230370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:07.231259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:07.231259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:07.231259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:07.231259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:07.235784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:07.237367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:07.362620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:07.366172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c17e1-c82c-4055-96e9-95e1a2ce9525 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.408117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.409021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.409470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.409845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.410694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:07.411429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:07.413302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:07.427387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:07.434704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:07.556149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4299d-e23e-4fc8-bcf4-a416fa3229b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.557729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:07.593284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.594091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.594376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.594646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.601984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:07.603418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:07.618614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:07.624794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:07.626316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:07.628759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:07.741930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef6ec5-58c0-4ce5-9d1a-cc61633c0c97 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.779276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.779974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.780234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.780540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.781402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:07.781718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:07.783167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:07.806101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:07.807339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:07.910187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27047a-34f6-47e1-8d5c-86fdf50b871f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:07.950156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:07.950813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:07.951130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:07.951370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:07.981622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:08.026916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:08.028382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:08.148475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:59:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:59:08.151483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:08.175432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:08.259171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856388-448c-402f-9f38-4013e9de4c1c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:08.296632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:08.297758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:08.298174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:08.298314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:08.310048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:08.312063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:08.332869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:08.333889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:08.335157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:08.335157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:08.335157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:08.335157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:08.335157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:08.340045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:08.407631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:08.451447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c398bd-2da6-4bdb-b7c9-7b3d6ffb1f10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:08.489951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:08.492465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:08.492465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:08.492465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:08.492792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:08.494040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:08.494500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:08.508209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:08.514278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:08.595406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:08.632845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63de41-4d9b-4cb8-9367-8c42a257c9c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:08.670329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:08.671139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:08.671423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:08.671696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:08.678959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:08.680468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:08.694644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:08.695753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:08.697349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:08.701797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:08.808909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022448a-f4a6-423e-bed3-389d4c5bb4b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:08.834729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:08.847424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:08.848094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:08.848378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:08.848686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:08.849479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:08.849788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:08.851448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ca99d-790f-4f77-b17b-e5627db415bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:08.863933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:08.869943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:08.977292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86024ca3-51db-42f7-88a2-034f23a909a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:09.006974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:09.013942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:09.015247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:09.015727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:09.016020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:09.016895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=beb0197dc69744d69c47ba294b9a4f41) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:09.017211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:09.018658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51ee1-3387-4ed4-a438-0d9e0392a7ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:09.032475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:59:08 2022] GET /identity/v3/role_assignments?user.id=beb0197dc69744d69c47ba294b9a4f41&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:11.046746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4cf17-d755-4bcb-a35b-76f94b51ee14 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.047334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4cf17-d755-4bcb-a35b-76f94b51ee14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.047334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4cf17-d755-4bcb-a35b-76f94b51ee14 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.048949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1220] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:11.055672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.056119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.056290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.075884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:11.077151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:11.153923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:11.177097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4cf17-d755-4bcb-a35b-76f94b51ee14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:11.295153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4cf17-d755-4bcb-a35b-76f94b51ee14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:11.330419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:11.331137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.331419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.331689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.341639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:11.349707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:11.351002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:11.357320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:59:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:59:11.358177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:11.479004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec5564-0838-4512-ae55-a813a3a7c122 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:11.508248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:11.520023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:11.520720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.521019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.521296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.522244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:11.522615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:11.524075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:11.533032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:59:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:11.539190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:11.587707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:11.651447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39414d55-5a0f-45b7-b450-4ee2432e5b0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:11.688341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:11.689178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.689461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.689737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.701168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:11.702628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:11.720198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:11.721382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:11.722406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:11.722406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:11.722406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:11.722406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:11.722406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:11.727725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:11.841135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138b63-10f4-4b68-875a-e1d8b191729e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:11.874564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:11.875288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:11.875511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:11.875781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:11.876853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:11.877158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:11.878575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:11.892511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:11.898906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.014494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:12.024894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70896068-1b3c-4837-b6f4-5739c64d13fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.066877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.067693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.067977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.068261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.076523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.078009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:12.095961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:12.097175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:12.098187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:12.103257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.159803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:12.232821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ed10f-fa4b-4fe1-9e15-be887b5839db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.267576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.268261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.268582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.268859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.269703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:12.270017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.271461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:12.289032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:12.294874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.405551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faae9e-a3a9-4363-bf45-bd2988b0f4c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.445390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.446360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.446606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.446884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.459169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.461403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:12.479899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:12.481144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:12.482282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:12.482282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:12.482282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:12.482282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:12.486057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:12.487153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.593542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed39aa1-b496-499f-a3ee-ef7e470f9c10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.596635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:12.636934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.637701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.637988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.638271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.639275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:12.639666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.641086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:12.653245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:12.659056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.669192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:12.768157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a2ae9-a344-414d-a182-08ae2e896ddd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.804880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.805759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.806049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.806361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.815562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.817333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:12.833170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:12.834504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:12.835591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:12.840787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:12.949571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca72bf8-527a-4c78-8946-8fc779dc112f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:12.989129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:12.989814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:12.990137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:12.990446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:12.991650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:12.992037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:12.993743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bddc71-722e-43ff-a74f-fe7aeafbe8f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:13.007679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:13.014103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:13.086964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:13.128050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2d82-22e4-4755-a638-7eec23bed6d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:13.177541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:13.178477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:13.178794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:13.179072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/beb0197dc69744d69c47ba294b9a4f41/roles/893fc5989bdf4eceb88313e63b81eb86` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:13.195346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=beb0197dc69744d69c47ba294b9a4f41, role_id=893fc5989bdf4eceb88313e63b81eb86)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:13.197043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e982ba-06c9-4fa7-85b2-c200b233f6f4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:13.211236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:59:13 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/beb0197dc69744d69c47ba294b9a4f41/roles/893fc5989bdf4eceb88313e63b81eb86 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:59:15.392739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc19d-7761-46c0-9343-ae3f6334fc6d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:15.393287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc19d-7761-46c0-9343-ae3f6334fc6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:15.393287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbc19d-7761-46c0-9343-ae3f6334fc6d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:15.395001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1233] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:15.400782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:15.401369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:15.401369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:15.419505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:15.420790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:15.493461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:15.519964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbc19d-7761-46c0-9343-ae3f6334fc6d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:15.550219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:15.635873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbc19d-7761-46c0-9343-ae3f6334fc6d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:15.669682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:15.670369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:15.670796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:15.671084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:15.686488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:15.694352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:15.695588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:15.696588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Tue Jun 21 02:59:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:15.701974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:15.750279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:15.808365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe236501-19db-44fd-888d-76d5d70a934d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:15.846373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:15.847040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:15.847321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:15.847602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:15.848358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:15.848722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:15.850164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:15.859981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Tue Jun 21 02:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:15.865849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:15.955649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:15.973627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff49d4-b142-4122-83c0-7a9ded41d96d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:16.007984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:16.008870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:16.009187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:16.009471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:16.020779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:16.022611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:16.041233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:16.042261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:16.043267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:16.043267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:16.043267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:16.043267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:16.043267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:16.048256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:16.156663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58cc12-24ba-44f8-80b1-5c54f0dfee5c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:16.167446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:16.196656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:16.197334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:16.197619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:16.197899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:16.198633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:16.198938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:16.200317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:16.212690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:16.218526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:16.320322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1321fd-4b9a-42c9-85e4-4979a0311704 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:16.356154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:16.357141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:16.357522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:16.357799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:16.360605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:16.366003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:16.367607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:16.382310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:16.383520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:16.385030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:59:16.389439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:16.492349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5031f-e27d-4ac8-8cec-cf909c94193f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:16.530536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:16.531219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:16.531492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:16.531772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:16.532636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:16.532945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:16.534425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:16.554017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:16.559873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:16.664789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335955eb-4e53-441b-ae84-0d9b3df26e4c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:16.699505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:16.700200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:16.700545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:16.700825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:16.767740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:16.776255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:16.777855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:16.887460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:59:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 02:59:16.888740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:16.916545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:16.995328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8ea6-0b2d-44c8-914d-cede038f68b2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:17.033831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:17.034482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:17.034756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:17.035033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:17.046329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:17.047800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:17.066610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:17.067842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:17.068995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:17.068995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:17.068995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:17.068995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:17.068995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:17.074126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:17.181880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:17.188021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c1b0-8ddf-4ba3-8ed5-4074d1b9ccf2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:17.226993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:17.227699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:17.227998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:17.228232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:17.229046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:17.229348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:17.230750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:17.243628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:17.249019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:17.341374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:17.370688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc82db6-3a23-415c-9347-7252ecc1aa6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:17.418896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:17.419791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:17.420070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:17.420373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:17.431270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:17.433190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:17.451883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:17.453497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:17.454431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:17.459223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:17.592168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d74a8-7a59-4484-8c60-20877a3a301c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:17.634267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:17.634962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:17.635285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:17.635540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:17.636344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:17.636660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:17.638144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bfbed-6b61-4bfc-9928-ab910a6dade0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:17.652064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:17.660584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:17.777516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14b9d0-4b99-4567-96f7-f84ce9f6e381 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:17.822705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:17.823414 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:17.823760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:17.824041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:17.825007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] RBAC: Adding query filter params (role.id=893fc5989bdf4eceb88313e63b81eb86, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=beb0197dc69744d69c47ba294b9a4f41) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:17.825375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:17.826887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64802b14-7e5d-4da8-ba59-70a73acbcefe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:17.839816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:59:17 2022] GET /identity/v3/role_assignments?user.id=beb0197dc69744d69c47ba294b9a4f41&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=893fc5989bdf4eceb88313e63b81eb86 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:19.992426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9d81-955c-4f33-b7ac-1e169d31216d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:19.993008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9d81-955c-4f33-b7ac-1e169d31216d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:19.993008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad9d81-955c-4f33-b7ac-1e169d31216d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:19.994449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1247] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:20.001250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.001763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.001864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:20.007711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:20.021536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:20.022841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:20.101282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:20.127268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad9d81-955c-4f33-b7ac-1e169d31216d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:20.148986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:20.242945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad9d81-955c-4f33-b7ac-1e169d31216d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:20.279049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:20.279873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.280146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.280443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:20.292337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:20.299997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:20.301150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:20.302212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:59:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:20.307399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:20.420265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381c350-5db2-49fc-bbba-c18a17b74fb8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:20.452595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:20.458140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:20.458819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.459102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.459585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:20.460235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:20.460570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:20.461978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:20.470408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:20.476986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:20.559595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:20.585115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434105d3-288b-4e71-b496-132b23484b5e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:20.619266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:20.620072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.620382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.620683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:20.634706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:20.636245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:20.655393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:20.656537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:20.657691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:20.657691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:20.657691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:20.657691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:20.657691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:20.663150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:20.765947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194e745-08e1-4d23-ac5b-04449e5af4a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:20.805831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:20.806621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.806944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.807248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:20.808089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:20.808470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:20.810189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:20.824673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:20.830540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:20.949603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a85c86-136a-425a-905f-9a93d9ff6d69 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:20.984456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:20.994237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:20.996206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:20.996542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:20.996889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:21.006082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:21.007735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:21.024059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:21.025209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:21.026142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:59:21.031535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:21.094387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:21.141371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a0f95-6286-415e-b56a-9f0e8944e7cb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:21.179014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:21.179699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:21.179981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:21.180261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:21.181143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:21.181452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:21.182895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:21.201803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:21.207831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:21.320740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc3db3-f4c9-4f7e-86e1-7f9bddac6e10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:21.360815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:21.361392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:21.361709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:21.361950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:21.419181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:21.434477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:21.436183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:21.544569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:59:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:59:21.550591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:21.665652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e543bd-ba52-423d-967f-112dfd2509ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:21.680907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:21.706076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:21.706932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:21.707257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:21.707532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:21.720384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:21.722194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:21.744946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:21.746146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:21.747249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:21.747249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:21.747249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:21.747249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:21.747249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:21.752673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:21.849287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:21.866355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc605535-0135-4d49-9064-4594a2cb116e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:21.910190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:21.910957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:21.911281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:21.911589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:21.912495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:21.912914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:21.914361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:21.929235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:21.935661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:22.049020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b8a7-df60-476f-a996-f45c669a0551 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:22.086741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:22.087516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:22.087798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:22.088074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:22.095317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:22.096953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:22.111646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:22.112860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:22.113781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:22.119325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:22.226598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afbf79-3b21-4524-bb67-9f35b8cc998c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:22.261972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:22.262665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:22.262951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:22.263226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:22.264091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:22.264501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:22.266121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7e241-6642-416c-bb29-4302d78aee41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:22.282304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:22.289193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:22.318858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:22.410521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98bd91-c0e5-445e-9e84-b8194d3deea7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:22.449743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:22.450406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:22.450696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:22.451001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:22.451850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=beb0197dc69744d69c47ba294b9a4f41) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:22.452150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:22.453625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199efe27-f504-46e3-8afd-9cfdec31fcd7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:22.459341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:59:22 2022] GET /identity/v3/role_assignments?user.id=beb0197dc69744d69c47ba294b9a4f41&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:24.436166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b25c-0cf9-45bb-b2d8-ee71f2d29124 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:24.437033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b25c-0cf9-45bb-b2d8-ee71f2d29124 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:24.437033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b25c-0cf9-45bb-b2d8-ee71f2d29124 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:24.439126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1261] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:24.445536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:24.446317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:24.446464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:24.473976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:24.475629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:24.558674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:24.581696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b25c-0cf9-45bb-b2d8-ee71f2d29124 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:24.598770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:24.699603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b25c-0cf9-45bb-b2d8-ee71f2d29124 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:24.734534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:24.735319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:24.735601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:24.735873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:24.747762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:24.755622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:24.756708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:24.757842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:24.763215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:24.782462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:24.867502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058087b-af83-4386-8da1-ddff06b7d293 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:24.902606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:24.903251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:24.903532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:24.903807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:24.904595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:24.904900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:24.906271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:24.914354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:24.920246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:24.995025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:25.030130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5904-0d63-40cc-805b-67e906dc3857 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.065001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.065783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.066067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.066341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.077357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.078819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:25.096267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:25.097303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:25.098572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:25.098572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:25.098572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:25.098572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:25.098572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:25.103647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:25.188634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:25.209053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4ab14-e907-471b-a982-58aae4a92f1d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.247127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.247721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.248007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.248287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.249049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:25.249355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.250746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:25.264185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:25.270755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:25.383963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769884d-2c91-462c-90e9-7da2a64d3230 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.403840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:25.421384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.422177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.422454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.422727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.430671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.432114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:25.446497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:25.448091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:25.449137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:25.453588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:25.563393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504ea0-117b-4e94-99bb-b65ab4c65348 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.601535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.602212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.602503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.602775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.603611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:25.603918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.605389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:25.614095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:25.623749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:25.630505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:25.750885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cd076-7245-43c8-9e64-7bc53f456810 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.801684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.802472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.802757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.803188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.814113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.815511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:25.832555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:25.833559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:25.834786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:25.834786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:25.834786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:25.834786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:25.834786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:25.841907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:25.958700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296019-6d8b-4b81-b00d-4d3105c3cb12 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:25.994524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:25.995190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:25.995474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:25.995760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:25.996547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:25.996858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:25.998244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:26.010275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:26.019756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:26.063833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:26.137681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b2ef0-812c-4cb3-80b0-eae44f2d3766 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:26.176917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:26.177727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:26.178013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:26.178290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:26.186046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:26.187511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:26.202246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:26.203275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:26.204334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:26.209682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:26.267765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:26.317048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707df5c-2bb4-4257-bf91-ebd08ca178bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:26.353337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:26.353989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:26.354272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:26.354551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:26.355343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:26.355657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:26.357138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679f3e-1d78-4425-a17f-8779e7124dc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:26.369259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:26.378517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:26.474765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:26.485524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a143-465e-4473-a57d-8f795d3417fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:26.523210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:26.524087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:26.524363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:26.524671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] PATH_INFO: `/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/beb0197dc69744d69c47ba294b9a4f41/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:26.526757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] RBAC: Authorizing `identity:create_grant(project_id=55a2591b7b2f4643b5e3599935ea51a6, user_id=beb0197dc69744d69c47ba294b9a4f41, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:26.528202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d624f4b-a8cf-48ae-8a4a-812db4f3f795 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:26.544060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Tue Jun 21 02:59:26 2022] PUT /identity/v3/projects/55a2591b7b2f4643b5e3599935ea51a6/users/beb0197dc69744d69c47ba294b9a4f41/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 02:59:28.560014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722e4b-1c8f-4972-916e-f5bce3eeeb6b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:28.561104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722e4b-1c8f-4972-916e-f5bce3eeeb6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:28.561104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722e4b-1c8f-4972-916e-f5bce3eeeb6b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:28.563133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1274] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:28.568470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:28.568916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:28.569017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:28.587925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:28.589130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:28.658247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:28.683404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722e4b-1c8f-4972-916e-f5bce3eeeb6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:28.800089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722e4b-1c8f-4972-916e-f5bce3eeeb6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:28.817917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:28.836136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:28.836976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:28.837259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:28.837532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:28.848022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 02:59:28.855948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:28.857120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:28.858048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Tue Jun 21 02:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:28.864905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:28.950020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:28.970276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ae88d-99c2-4c3d-944e-a7b9eae39597 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.005945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.006614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.006909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.007197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.008140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:29.008439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.009865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:29.018210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Tue Jun 21 02:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:29.024131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:29.127389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aed81d-a852-40d9-8ce2-52d73325913b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.161774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.162591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.162876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.163150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.174295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.175760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:29.193045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:29.194068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:29.195245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:29.195245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:29.195245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:29.195245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:29.195245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 02:59:29.200331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:29.305004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc79e2-422c-4f29-b7f2-71ae9d95a370 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.339915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.340592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.340876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.341150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.341870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:29.342176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.343548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:29.355466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:29.364475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:29.399594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:29.469498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ab8cc-581b-4e5d-9c9a-4d972a65006a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.504973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.505769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.506053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.506334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.514191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.515634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 02:59:29.529889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:29.530992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:29.532033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:29.536970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:29.541213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:29.641245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779825d-90cb-45f6-ada0-4c3aecf0ce45 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.676786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.677381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.677661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.677932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.678766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:29.679096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.680686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:29.697657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Tue Jun 21 02:59:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:29.703735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:29.808679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:29.818783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43889bf0-fc52-4cc0-8809-707367612e32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:29.853875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:29.854534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:29.854814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:29.855085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:29.927829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:29.929906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.021376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:30.033550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Tue Jun 21 02:59:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 02:59:30.043697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:30.114676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:30.158363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d0199-688b-4250-9a0e-c87c1f67679b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:30.201368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:30.202103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:30.202386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:30.202660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:30.217403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:30.218870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 02:59:30.237578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:30.238628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:30.239749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 02:59:30.239749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 02:59:30.239749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 02:59:30.239749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:30.239749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 02:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:30.248092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:30.355666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64731a61-d4ee-48f1-ade1-59339e65ceee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:30.392446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:30.393119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:30.393403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:30.393680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:30.394403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:30.394710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:30.396084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.407525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Tue Jun 21 02:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:30.413505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:30.525088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dcbd1-c84c-4468-a905-66667b70b8d1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:30.540626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:30.563425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:30.564215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:30.564529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:30.564808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:30.572013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:30.573520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 02:59:30.588114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:30.589167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 21 02:59:30.590183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Tue Jun 21 02:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:30.595501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:30.649421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:30.701100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86710e-b41c-4171-a334-129f7fdebfc6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:30.736334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:30.737006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:30.737287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:30.737555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:30.738368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:30.738675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:30.740089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa4259-89f0-4557-b8f7-0aaf0130f7fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.752594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Tue Jun 21 02:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:30.759317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:30.880723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36034e-d24c-47a3-9de9-9cfdc47060f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:30.924173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:30.924945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:30.925342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:30.925745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:30.926762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] RBAC: Adding query filter params (role.id=60c47ed962914bf8985b2ab9f7d94f07, scope.project.id=55a2591b7b2f4643b5e3599935ea51a6, user.id=beb0197dc69744d69c47ba294b9a4f41) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:30.927109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:30.928944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bd2ac-b079-41f4-92e3-da5189bdf3b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:30.936899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Tue Jun 21 02:59:30 2022] GET /identity/v3/role_assignments?user.id=beb0197dc69744d69c47ba294b9a4f41&scope.project.id=55a2591b7b2f4643b5e3599935ea51a6&role.id=60c47ed962914bf8985b2ab9f7d94f07 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:32.953514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75fe9-77c8-4501-9747-4a98a289dcbc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:32.954107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75fe9-77c8-4501-9747-4a98a289dcbc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:32.954107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af75fe9-77c8-4501-9747-4a98a289dcbc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:32.956172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1288] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:32.962608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:32.963009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:32.963167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:32.982175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:32.983388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:33.016636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:33.060398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:33.083186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75fe9-77c8-4501-9747-4a98a289dcbc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:33.195685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af75fe9-77c8-4501-9747-4a98a289dcbc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:33.206423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:33.232300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:33.233140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:33.233417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:33.233695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:33.234370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:33.235756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:33.255969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:33.257286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:33.258198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:59:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:33.263815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:33.369821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ada4e-fc6f-4c1c-b602-0075197d6ed3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:33.405232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:33.405887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:33.406171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:33.406446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:33.407156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:33.407464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:33.408893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bb810-0dbb-41f2-85f8-464ebd59ea77 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:33.414001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jun 21 02:59:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:33.419976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:33.522041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c175ee-7ae4-4359-8bc4-9cc173f4526f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:33.557146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:33.557801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:33.558083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:33.558361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:33.559067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:33.559373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:33.560767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760ddfd-34c2-44db-a49c-dbd67f599a23 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:33.565991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jun 21 02:59:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:35.636884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca31bf3-2354-4dc4-8760-b99b29c319ca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:35.637466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca31bf3-2354-4dc4-8760-b99b29c319ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:35.637466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca31bf3-2354-4dc4-8760-b99b29c319ca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:35.639604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1293] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:35.648574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33e95-c1ed-4e80-b8c1-31b978f64d84 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:35.648995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33e95-c1ed-4e80-b8c1-31b978f64d84 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:35.649305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c33e95-c1ed-4e80-b8c1-31b978f64d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:35.672649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30c33e95-c1ed-4e80-b8c1-31b978f64d84 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:35.673856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30c33e95-c1ed-4e80-b8c1-31b978f64d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:35.707024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:35.746435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:35.770337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca31bf3-2354-4dc4-8760-b99b29c319ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:35.805374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:35.896906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca31bf3-2354-4dc4-8760-b99b29c319ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:35.936333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:35.937044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:35.937333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:35.937624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:35.938449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:35.939999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69574573-8b8e-4b9c-8753-77518c47052d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:35.955264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Tue Jun 21 02:59:35 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:59:37.955191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379c63-cb17-4d14-af6e-9a8a8a236c6b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:37.955784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379c63-cb17-4d14-af6e-9a8a8a236c6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:37.955784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4379c63-cb17-4d14-af6e-9a8a8a236c6b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:37.957169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1296] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:37.962390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:37.962815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:37.962910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:37.980376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:37.981588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:38.060386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:38.083537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379c63-cb17-4d14-af6e-9a8a8a236c6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:38.203857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4379c63-cb17-4d14-af6e-9a8a8a236c6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:38.246628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:38.247382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:38.247664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:38.247942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:38.248636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:38.250012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:38.262757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:38.263956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:38.264965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:59:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:38.270939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:38.291487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:38.375055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d36b9-8e44-49da-a799-3b1ec466530d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:38.410626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:38.411279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:38.411561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:38.411833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:38.412574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:38.412882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:38.414372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:38.419699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jun 21 02:59:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 02:59:38.426581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:38.462680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:38.538715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e6e0c-e9c8-44e7-87c3-a3b6603f6104 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:38.575669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:38.576311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:38.576622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:38.576891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:38.577679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] RBAC: Adding query filter params (interface=public, service_id=cc8b23deaaf94d72b5109426ca0bb839, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:38.577987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:38.579357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965e599-7dc4-44b6-9187-dd5108d1956c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:38.584901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Tue Jun 21 02:59:38 2022] GET /identity/v3/endpoints?service_id=cc8b23deaaf94d72b5109426ca0bb839&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:38.590820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:38.675994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:38.697334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc734454-5245-4ee0-bfea-a4d93e91605a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:38.732139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:38.732830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:38.733112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:38.733383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:38.734065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:38.735449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8330cc-468c-405b-823f-c4aee2f4363d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:38.742430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Tue Jun 21 02:59:38 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 02:59:40.731477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f81bd-6814-4b2b-8c97-44223ccaec68 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:40.732376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f81bd-6814-4b2b-8c97-44223ccaec68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:40.732376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f81bd-6814-4b2b-8c97-44223ccaec68 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:40.734538 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1302] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 02:59:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 02:59:40.740892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:40.741365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:40.741461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:40.764944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 02:59:40.766150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 02:59:40.835410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 02:59:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 02:59:40.865378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f81bd-6814-4b2b-8c97-44223ccaec68 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:40.978183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:40.985362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f81bd-6814-4b2b-8c97-44223ccaec68 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:41.019719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:41.020536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:41.020817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:41.021089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:41.021748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:41.023278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 02:59:41.035099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 02:59:41.036206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 21 02:59:41.037124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Tue Jun 21 02:59:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 02:59:41.043274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:41.129903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:41.159273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb72fb0-bbf2-4d1c-ba86-69a24d970082 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:41.194108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:41.194790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:41.195150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:41.195445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:41.196162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 02:59:41.196509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:41.197901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f51003-0be8-41c1-bdbc-304b07d55d5c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:41.203260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Tue Jun 21 02:59:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 02:59:41.209251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 02:59:41.329639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0af65e-a7ed-4447-8dbf-559922d830d7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 02:59:41.367921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 02:59:41.368598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 02:59:41.368881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 02:59:41.369158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 02:59:41.369900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 02:59:41.371267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24da665-9258-4f06-af47-a5f254f00cac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 02:59:41.397076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Tue Jun 21 02:59:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 02:59:51.199973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=30 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 02:59:51.388932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:44.400816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1dee5-4553-4f26-8441-712117949960 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:44.401403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1dee5-4553-4f26-8441-712117949960 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:44.401643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1dee5-4553-4f26-8441-712117949960 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:44.408301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0c96c-9280-423c-9a1e-260ecd194903 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:44.408778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0c96c-9280-423c-9a1e-260ecd194903 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:44.409116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0c96c-9280-423c-9a1e-260ecd194903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:44.412976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1307] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:00:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:44.491447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5b0c96c-9280-423c-9a1e-260ecd194903 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:44.492682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5b0c96c-9280-423c-9a1e-260ecd194903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:44.603171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:00:44.653550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:44.654025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:44.654485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:44.656758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1309] 10.222.0.16 () {54 vars in 964 bytes} [Tue Jun 21 03:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:44.662233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ec3b-38ba-4a69-8d65-d4d8458fe77c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:44.662583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ec3b-38ba-4a69-8d65-d4d8458fe77c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:44.662763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18ec3b-38ba-4a69-8d65-d4d8458fe77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:44.817189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6e18ec3b-38ba-4a69-8d65-d4d8458fe77c None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:44.818610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e18ec3b-38ba-4a69-8d65-d4d8458fe77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:44.840440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:44.901687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Jun 21 03:00:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:00:44.910643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:00:44.912434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:45.029801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed78d88-0f69-414c-bca6-7d6c39a9b7c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:00:45.108602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:00:45.109434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:45.109796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:45.110210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:45.111993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:45.176730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:45.236685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:00:45.238125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e03ae6-0497-4995-9331-e66c05434850 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:00:45.258716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:00:47.538202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7027b-c100-4b4b-891c-d9d564b807a4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:47.538763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7027b-c100-4b4b-891c-d9d564b807a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:47.538763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7027b-c100-4b4b-891c-d9d564b807a4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:47.540302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1312] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:00:47.545449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1687b7-6727-4d0f-b2fe-a8823092e4ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:47.545903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1687b7-6727-4d0f-b2fe-a8823092e4ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:47.545985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1687b7-6727-4d0f-b2fe-a8823092e4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:47.617616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c1687b7-6727-4d0f-b2fe-a8823092e4ab None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:47.618852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c1687b7-6727-4d0f-b2fe-a8823092e4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:47.720197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:00:47.774396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:47.774677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:47.774962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:47.776709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1314] 10.222.0.16 () {54 vars in 964 bytes} [Tue Jun 21 03:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:47.783064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d789d-cd53-40d0-837a-9900be8f26cc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:47.783365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d789d-cd53-40d0-837a-9900be8f26cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:47.783652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d789d-cd53-40d0-837a-9900be8f26cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:47.794550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:47.864635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f04d789d-cd53-40d0-837a-9900be8f26cc None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:47.866251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f04d789d-cd53-40d0-837a-9900be8f26cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:47.968985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1315] 10.222.0.16 () {58 vars in 1061 bytes} [Tue Jun 21 03:00:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:00:47.975089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:00:47.976730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:48.097113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d0fb5-b9f0-4a22-ae14-ff9d8487967b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:00:48.176572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:48.179503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:00:48.180234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:48.180640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:48.180983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:48.183325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:48.302308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:00:48.303810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f02c2-70aa-4b54-a543-bd765ac3cf2b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:00:48.323733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:00:50.853019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:50.853678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:50.853678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:50.854985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1317] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:50.861274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87539-a781-40e0-8439-d87540cd4bec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:50.861778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87539-a781-40e0-8439-d87540cd4bec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:50.861883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87539-a781-40e0-8439-d87540cd4bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:50.937066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15c87539-a781-40e0-8439-d87540cd4bec None None] MFA Rules not processed for user `9001fa39b72641f0a1cf1d17e323ecc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:50.938284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15c87539-a781-40e0-8439-d87540cd4bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:50.994389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:51.040467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1318] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:00:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:00:51.092237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:00:51.094536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:51.207700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8f1b5-6e00-4ee5-96d9-51a6e825e33b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:00:51.210532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:51.287222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:00:51.287864 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:51.288146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:51.288431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:51.290080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:51.404157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:00:51.405658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b718d8f-159e-47e5-8f2e-58a886b45be9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:00:51.424777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:00:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:00:54.172243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:54.172819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:54.172819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:54.174222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1320] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:54.179803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d8800-6b00-40b0-a0f5-d6e03fb085fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:54.180255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d8800-6b00-40b0-a0f5-d6e03fb085fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:54.180356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d8800-6b00-40b0-a0f5-d6e03fb085fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:54.221503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:54.248735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ed8d8800-6b00-40b0-a0f5-d6e03fb085fd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:54.249939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ed8d8800-6b00-40b0-a0f5-d6e03fb085fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:54.350111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1321] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:00:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:00:54.393666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:00:54.396161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:54.507472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a13d1-c6dc-4e38-9184-9cc6ffd1948e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:00:54.583495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:00:54.584143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:54.584466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:54.584728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:54.586369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:54.594224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:54.706156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:00:54.707678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19247-7ba3-429c-9761-dc66d4389ace service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:00:54.727195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:00:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:00:57.463375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:57.463899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:57.463899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:57.465410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1323] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:00:57.472170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3bd2-0218-4441-896f-c4a414e5276f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:57.472711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3bd2-0218-4441-896f-c4a414e5276f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:57.472711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3bd2-0218-4441-896f-c4a414e5276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:57.540174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-520d3bd2-0218-4441-896f-c4a414e5276f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:00:57.541415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-520d3bd2-0218-4441-896f-c4a414e5276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:57.604609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:57.650093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:00:57.686496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:00:57.688045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:57.798837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cb147-61d2-4c63-896d-3f4392c1ae07 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:00:57.811241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:00:57.874706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:00:57.875354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:00:57.875636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:00:57.876195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:00:57.877885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:00:57.988828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:00:57.990335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f60d9f-b46c-4951-97e7-3ba17ac5edf7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:00:58.010093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:00.058194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:00.058802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:00.058802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:00.059965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1326] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:00.065414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a459-26ef-4487-8e3c-ed4cc7b2782f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:00.065993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a459-26ef-4487-8e3c-ed4cc7b2782f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:00.066151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7a459-26ef-4487-8e3c-ed4cc7b2782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:00.114618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:00.134758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1a7a459-26ef-4487-8e3c-ed4cc7b2782f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:00.135994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1a7a459-26ef-4487-8e3c-ed4cc7b2782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:00.244105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:00.279562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:00.281243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:00.393777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d166d4-0fc9-45aa-9418-b3c09fae820a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:00.478361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:00.479025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:00.479313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:00.479591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:00.481357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:00.499897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:00.595799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:00.597254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d5c2f-c0df-4616-9ae2-5fed6400fed4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:00.624541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:02.681772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:02.682326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:02.682326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:02.683693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1329] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:02.689008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03635f32-5a9b-487f-a5d7-e00ef60e9a0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:02.689465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03635f32-5a9b-487f-a5d7-e00ef60e9a0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:02.689655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03635f32-5a9b-487f-a5d7-e00ef60e9a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:02.757648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-03635f32-5a9b-487f-a5d7-e00ef60e9a0a None None] MFA Rules not processed for user `9001fa39b72641f0a1cf1d17e323ecc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:02.758876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03635f32-5a9b-487f-a5d7-e00ef60e9a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:02.831151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:02.867604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:02.905306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:02.906920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:03.015281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ceee3-ebe3-4615-92c7-48b62710f3a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:03.035885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:03.093225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:03.093878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:03.094164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:03.094439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:03.096075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:03.223814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:03.225307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500a6f3-de8d-4481-93f7-2ab90baefefb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:03.248447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:05.497387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:05.497913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:05.497913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:05.499332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1332] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:05.507224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412992b7-bbfe-43ac-9308-c44082f0649f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:05.507697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412992b7-bbfe-43ac-9308-c44082f0649f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:05.507825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412992b7-bbfe-43ac-9308-c44082f0649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:05.576718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:05.579501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-412992b7-bbfe-43ac-9308-c44082f0649f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:05.581279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-412992b7-bbfe-43ac-9308-c44082f0649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:05.685097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:05.722727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:05.724361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:05.837802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47eb52-8148-406a-8fcd-d067b7bc205c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:05.933792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:05.934458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:05.934744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:05.935021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:05.936733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:05.965146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:06.050394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:06.051914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8c9ea-40f6-4fc9-b1e3-6b01c4d79ad8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:06.070566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:01:08.116938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:08.117779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:08.117779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:08.119970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1335] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:08.125964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa056c9-1c32-422e-8706-2e3a13e79082 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:08.126408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa056c9-1c32-422e-8706-2e3a13e79082 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:08.126502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa056c9-1c32-422e-8706-2e3a13e79082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:08.194392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6fa056c9-1c32-422e-8706-2e3a13e79082 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:08.195604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6fa056c9-1c32-422e-8706-2e3a13e79082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:08.280829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:08.308155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:08.348880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:08.350460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:08.463152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1ddb1-e5aa-4392-80fd-4bd11166c75a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:08.500253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:08.548047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:08.548726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:08.549009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:08.549276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:08.550966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:08.670804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:08.672253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835db637-93d9-405e-9f90-866764b1e762 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:08.699532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:11.078887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:11.079493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:11.079493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:11.080944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1338] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:11.087635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e513b-5758-4501-8496-9274d4b26f50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:11.088399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e513b-5758-4501-8496-9274d4b26f50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:11.088746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e513b-5758-4501-8496-9274d4b26f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:11.168521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-af1e513b-5758-4501-8496-9274d4b26f50 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:11.169761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af1e513b-5758-4501-8496-9274d4b26f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:11.178897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:11.283446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:11.319269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:11.320847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:11.443464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78335a07-0fe0-43e4-a616-13179ff2cd1b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:11.521748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:11.522385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:11.522676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:11.522942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:11.524626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:11.564229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:11.641327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:11.642785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a25a9-2937-4fcb-a194-499e0da90cab service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:11.663897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:15.850661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:15.851212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:15.851212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:15.852600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1341] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:15.858059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ffa0b-07a2-445e-93c0-f73de332bb22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:15.858480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ffa0b-07a2-445e-93c0-f73de332bb22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:15.858625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ffa0b-07a2-445e-93c0-f73de332bb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:15.934502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2c4ffa0b-07a2-445e-93c0-f73de332bb22 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:15.936114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2c4ffa0b-07a2-445e-93c0-f73de332bb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:16.039424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:16.054305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:01:16.093124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:16.095695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:16.202997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edae29-bc2f-4d89-9074-7aae51474a5b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:16.207679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:16.209140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:16.209508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:16.209922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:16.222182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:16.225274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2135561-ba28-4be2-ab88-d50fadf6a4a6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:16.235387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:18.885181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:18.885994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:18.885994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:18.887498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1344] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:18.894995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf664-8bcc-429b-9588-8b0db6789dd2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:18.895753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf664-8bcc-429b-9588-8b0db6789dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:18.896042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cf664-8bcc-429b-9588-8b0db6789dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:18.930737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-084cf664-8bcc-429b-9588-8b0db6789dd2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:18.932716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-084cf664-8bcc-429b-9588-8b0db6789dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:18.967948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:19.034897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:19.074056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:19.075941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:19.090220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059947bc-84c0-449b-90b8-73b7039aeb28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:19.093933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:19.094595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:19.094880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:19.095153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:19.106618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:19.108126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa4791-a93e-49dd-bc65-581f7ae8324c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:19.127901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:23.341852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714b42-0dce-4b1e-abfd-85909cdee348 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:23.342394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714b42-0dce-4b1e-abfd-85909cdee348 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:23.342394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82714b42-0dce-4b1e-abfd-85909cdee348 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:23.343833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1347] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:23.349956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c80033-50bb-4bf9-8b1f-47e35f6b3752 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:23.350430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c80033-50bb-4bf9-8b1f-47e35f6b3752 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:23.350572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c80033-50bb-4bf9-8b1f-47e35f6b3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:23.369706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f7c80033-50bb-4bf9-8b1f-47e35f6b3752 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:23.371429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7c80033-50bb-4bf9-8b1f-47e35f6b3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:23.381837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:01:23.422143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82714b42-0dce-4b1e-abfd-85909cdee348 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:23.434215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82714b42-0dce-4b1e-abfd-85909cdee348 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:23.438776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:23.439430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:23.439715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:23.440109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:23.450577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:23.452037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd05bff-5d83-4ff8-9f28-2679c7ae8ba1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:23.462932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:28.433966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648e76-2f00-48aa-aba3-b36dc14699f7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:28.435322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648e76-2f00-48aa-aba3-b36dc14699f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:28.435827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648e76-2f00-48aa-aba3-b36dc14699f7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:28.437460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1350] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:01:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:28.446021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306083f-6721-4eb2-8e0c-ed09709c6438 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:28.446818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306083f-6721-4eb2-8e0c-ed09709c6438 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:28.447250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6306083f-6721-4eb2-8e0c-ed09709c6438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:28.516613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6306083f-6721-4eb2-8e0c-ed09709c6438 None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:28.518067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6306083f-6721-4eb2-8e0c-ed09709c6438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:28.590677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:01:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:32.680603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea58b8-53e9-4ec5-9172-17d93b38fa0a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:32.681159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea58b8-53e9-4ec5-9172-17d93b38fa0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:32.681159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea58b8-53e9-4ec5-9172-17d93b38fa0a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:32.682646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1352] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:32.689001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87d53-3b14-4bd4-b1da-6e39f3ca2ed2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:32.689968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87d53-3b14-4bd4-b1da-6e39f3ca2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:32.690463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87d53-3b14-4bd4-b1da-6e39f3ca2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:32.709660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ad87d53-3b14-4bd4-b1da-6e39f3ca2ed2 None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:32.710855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ad87d53-3b14-4bd4-b1da-6e39f3ca2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:32.720114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:01:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:33.364489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=13 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:34.137848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2164-be26-42a7-9ae9-3ea1c16e5e53 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:34.138543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2164-be26-42a7-9ae9-3ea1c16e5e53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:34.138543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2164-be26-42a7-9ae9-3ea1c16e5e53 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:34.139967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1354] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:34.145086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4577f9-ca8a-4ae5-9077-fd18f4ff320e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:34.145543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4577f9-ca8a-4ae5-9077-fd18f4ff320e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:34.146009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4577f9-ca8a-4ae5-9077-fd18f4ff320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:34.165408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db4577f9-ca8a-4ae5-9077-fd18f4ff320e None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:34.166652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db4577f9-ca8a-4ae5-9077-fd18f4ff320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:34.176891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:01:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:37.935255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e215-cb7d-4635-9651-7a12fa9d3bc2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:37.935809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e215-cb7d-4635-9651-7a12fa9d3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:37.935809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e215-cb7d-4635-9651-7a12fa9d3bc2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:37.937714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1356] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:37.943562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b909-7a1a-454f-b911-031222747f81 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:37.944473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b909-7a1a-454f-b911-031222747f81 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:37.945027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2b909-7a1a-454f-b911-031222747f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:37.968675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9cc2b909-7a1a-454f-b911-031222747f81 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:37.970135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9cc2b909-7a1a-454f-b911-031222747f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:37.979915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:38.043618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14d3e0-9f15-4bd9-b24c-b38cdc7faa4c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:38.044327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14d3e0-9f15-4bd9-b24c-b38cdc7faa4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:38.044863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14d3e0-9f15-4bd9-b24c-b38cdc7faa4c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:38.046819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1358] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jun 21 03:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:38.059023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a448b94-a7a8-4ca8-81ee-84ce6b61ff5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:38.059310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a448b94-a7a8-4ca8-81ee-84ce6b61ff5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:38.059786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a448b94-a7a8-4ca8-81ee-84ce6b61ff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:38.105635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:38.114568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a448b94-a7a8-4ca8-81ee-84ce6b61ff5c None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:38.115760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a448b94-a7a8-4ca8-81ee-84ce6b61ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:38.213460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jun 21 03:01:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:38.221845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14d3e0-9f15-4bd9-b24c-b38cdc7faa4c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:38.239223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14d3e0-9f15-4bd9-b24c-b38cdc7faa4c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:38.245174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:38.246051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:38.246483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:38.247257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:38.257726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:38.259675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdc15c-0dcc-43f0-80f8-df98a9d3ef2e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:38.285488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:01:40.553319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e8f-45bc-420a-851e-e4bea63c2d22 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:40.553905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e8f-45bc-420a-851e-e4bea63c2d22 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:40.553905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d68e8f-45bc-420a-851e-e4bea63c2d22 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:40.556786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1361] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:40.560567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99a6f-8e12-4d17-b58e-eec98a3295fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:40.561026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99a6f-8e12-4d17-b58e-eec98a3295fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:40.561026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99a6f-8e12-4d17-b58e-eec98a3295fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:40.578923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92c99a6f-8e12-4d17-b58e-eec98a3295fd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:40.580126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92c99a6f-8e12-4d17-b58e-eec98a3295fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:40.588785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:40.632740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68e8f-45bc-420a-851e-e4bea63c2d22 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:40.648180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d68e8f-45bc-420a-851e-e4bea63c2d22 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:40.653901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:40.654556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:40.654844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:40.655124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:40.666796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:40.668442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20725aca-294b-467a-ae67-6a529aa539ed service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:40.679173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:43.024694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8e52-16ed-4472-998c-727994690308 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:43.025215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8e52-16ed-4472-998c-727994690308 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:43.025215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f8e52-16ed-4472-998c-727994690308 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:43.026626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1364] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:43.031998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f22d7-81ca-47e4-b617-565316981336 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:43.032784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f22d7-81ca-47e4-b617-565316981336 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:43.032784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f22d7-81ca-47e4-b617-565316981336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:43.051468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc7f22d7-81ca-47e4-b617-565316981336 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:43.052787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc7f22d7-81ca-47e4-b617-565316981336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:43.060893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:43.105223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5c41c-a037-4c9c-930d-5497c2a25604 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:43.105542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5c41c-a037-4c9c-930d-5497c2a25604 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:43.105821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5c41c-a037-4c9c-930d-5497c2a25604 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:43.107178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1366] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jun 21 03:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:43.112628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aecc3-c159-47a5-81eb-6ca833f1dab1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:43.113177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aecc3-c159-47a5-81eb-6ca833f1dab1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:43.113645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aecc3-c159-47a5-81eb-6ca833f1dab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:43.132954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d36aecc3-c159-47a5-81eb-6ca833f1dab1 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:43.134388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d36aecc3-c159-47a5-81eb-6ca833f1dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:43.144358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jun 21 03:01:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:43.152370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5c41c-a037-4c9c-930d-5497c2a25604 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:43.167359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5c41c-a037-4c9c-930d-5497c2a25604 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:43.172752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:43.173429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:43.173713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:43.173981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:43.182343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:43.183928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fcf22-1f36-4d32-a6f8-67fc665d3e6b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:43.193589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:45.279167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0954d-01f6-46f2-8f23-afcaf81e645b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:45.279751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0954d-01f6-46f2-8f23-afcaf81e645b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:45.279751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0954d-01f6-46f2-8f23-afcaf81e645b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:45.281339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1369] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:45.286580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dce0d-68f2-481f-a7b6-0170b46fab14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:45.287023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dce0d-68f2-481f-a7b6-0170b46fab14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:45.287244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278dce0d-68f2-481f-a7b6-0170b46fab14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:45.306046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-278dce0d-68f2-481f-a7b6-0170b46fab14 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:45.307462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-278dce0d-68f2-481f-a7b6-0170b46fab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:45.317130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:45.365510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0954d-01f6-46f2-8f23-afcaf81e645b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:45.381334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0954d-01f6-46f2-8f23-afcaf81e645b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:45.387093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:45.387894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:45.388278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:45.388684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:45.398229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:45.400074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678b325-46bf-4a19-b052-2dd979edcb43 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:45.412265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:45.942532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3398-43d2-48e9-9846-dfc7b46605db None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:45.943465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3398-43d2-48e9-9846-dfc7b46605db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:45.944128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d3398-43d2-48e9-9846-dfc7b46605db None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:45.946163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1372] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:45.951659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cec52-23c0-43e7-8a09-0844d59cdcab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:45.952702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cec52-23c0-43e7-8a09-0844d59cdcab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:45.953222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cec52-23c0-43e7-8a09-0844d59cdcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:45.975347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f23cec52-23c0-43e7-8a09-0844d59cdcab None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:45.976958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f23cec52-23c0-43e7-8a09-0844d59cdcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:45.986697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:46.339635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad54504-16a4-4a33-8e59-86fa0c49c6ee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:46.341305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad54504-16a4-4a33-8e59-86fa0c49c6ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:46.341802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad54504-16a4-4a33-8e59-86fa0c49c6ee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:46.344284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1374] 10.222.0.16 () {54 vars in 963 bytes} [Tue Jun 21 03:01:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:46.350099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3bf0-cc7e-4238-8f98-c93fed61a68c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:46.350494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3bf0-cc7e-4238-8f98-c93fed61a68c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:46.350494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f3bf0-cc7e-4238-8f98-c93fed61a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:46.404965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-551f3bf0-cc7e-4238-8f98-c93fed61a68c None None] MFA Rules not processed for user `18bc17792b4d4879a1fd533206245694`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:46.406221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-551f3bf0-cc7e-4238-8f98-c93fed61a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:46.469268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Tue Jun 21 03:01:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:46.477714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad54504-16a4-4a33-8e59-86fa0c49c6ee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:46.491548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad54504-16a4-4a33-8e59-86fa0c49c6ee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:46.495559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:46.496754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:46.497713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:46.498206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:46.512605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:46.514376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8d0e3-9fe0-40dc-ae9f-6d1d9b0b6ac0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:46.527338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:47.459219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0851a-b522-4679-862c-a6f035eae262 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:47.459764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0851a-b522-4679-862c-a6f035eae262 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:47.459764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0851a-b522-4679-862c-a6f035eae262 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:47.461313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1377] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:47.466726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c77098-6093-4cea-8437-63283a92af1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:47.467190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c77098-6093-4cea-8437-63283a92af1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:47.467284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c77098-6093-4cea-8437-63283a92af1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:47.486767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-12c77098-6093-4cea-8437-63283a92af1c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:47.488546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12c77098-6093-4cea-8437-63283a92af1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:47.498274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:47.542868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0851a-b522-4679-862c-a6f035eae262 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:47.557996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0851a-b522-4679-862c-a6f035eae262 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:47.563639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:47.564283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:47.564602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:47.564885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:47.573285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:47.574887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c213033-b398-4109-8a19-0c1caadb57ac service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:47.578723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:01:47.586186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:49.673138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9924b-da5f-48d3-9e10-470a93ed461c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:49.673691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9924b-da5f-48d3-9e10-470a93ed461c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:49.673691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9924b-da5f-48d3-9e10-470a93ed461c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:49.675245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1380] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:49.680747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2092-83ce-475c-9899-a91f5bafd690 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:49.681168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2092-83ce-475c-9899-a91f5bafd690 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:49.681323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a2092-83ce-475c-9899-a91f5bafd690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:49.700290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a55a2092-83ce-475c-9899-a91f5bafd690 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:49.701534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a55a2092-83ce-475c-9899-a91f5bafd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:49.715896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:49.755220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9924b-da5f-48d3-9e10-470a93ed461c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:49.771110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9924b-da5f-48d3-9e10-470a93ed461c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:49.777181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:49.777837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:49.778128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:49.778412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:49.786960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:49.788682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b27d2-c051-4c7a-a15b-52e03ce0fee5 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:49.804024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:51.923281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bce12-38cd-4468-9ba7-74dad1a1b7bf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:51.923805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bce12-38cd-4468-9ba7-74dad1a1b7bf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:51.923805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bce12-38cd-4468-9ba7-74dad1a1b7bf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:51.925532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1383] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:51.930654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d634-b529-40c1-b47b-096afde9119c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:51.931174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d634-b529-40c1-b47b-096afde9119c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:51.931271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73d634-b529-40c1-b47b-096afde9119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:51.952281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ce73d634-b529-40c1-b47b-096afde9119c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:51.953573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce73d634-b529-40c1-b47b-096afde9119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:51.961922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:52.006119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bce12-38cd-4468-9ba7-74dad1a1b7bf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:52.022612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bce12-38cd-4468-9ba7-74dad1a1b7bf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:52.029193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:52.029872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:52.030156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:52.030432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:52.040730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:52.042360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b133-aa72-4e4f-8d3c-2c2d39feb389 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:52.053065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:54.134653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafedf6-d396-492c-b04c-afba705f05a8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:54.135652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafedf6-d396-492c-b04c-afba705f05a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:54.135652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafedf6-d396-492c-b04c-afba705f05a8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:54.136844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1386] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:01:54.142330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9fb6-ba7b-4203-a7aa-96adc8fd81de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:54.143094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9fb6-ba7b-4203-a7aa-96adc8fd81de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:54.143094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9fb6-ba7b-4203-a7aa-96adc8fd81de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:54.163581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-670c9fb6-ba7b-4203-a7aa-96adc8fd81de None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:54.164909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-670c9fb6-ba7b-4203-a7aa-96adc8fd81de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:54.174072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:54.226895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafedf6-d396-492c-b04c-afba705f05a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:54.253709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafedf6-d396-492c-b04c-afba705f05a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:54.263868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:54.265099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:54.265682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:54.266134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:54.282198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:54.285379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cef99-1b13-4277-87f0-5dcd33e76243 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:54.303992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:56.362029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7e45-1516-4117-8bdc-ac4a3c22bc84 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:56.362616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7e45-1516-4117-8bdc-ac4a3c22bc84 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:56.362829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7e45-1516-4117-8bdc-ac4a3c22bc84 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:56.364577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1389] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:56.371397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b824-ae0d-463e-b4b9-d173b290682e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:56.372301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b824-ae0d-463e-b4b9-d173b290682e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:56.372753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900b824-ae0d-463e-b4b9-d173b290682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:56.406134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9900b824-ae0d-463e-b4b9-d173b290682e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:56.407410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9900b824-ae0d-463e-b4b9-d173b290682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:56.421342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:01:56.466447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7e45-1516-4117-8bdc-ac4a3c22bc84 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:56.483609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7e45-1516-4117-8bdc-ac4a3c22bc84 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:56.490840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:56.491523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:56.491819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:56.492100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:56.502210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:56.504099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119061b6-a7bd-4e1c-b364-f964e518e2cd service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:56.514715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:01:58.592716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe74f-5ae0-4adc-aef3-c1aabb980e42 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:58.593285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe74f-5ae0-4adc-aef3-c1aabb980e42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:58.593285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77fe74f-5ae0-4adc-aef3-c1aabb980e42 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:58.594926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1392] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:01:58.601360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7abcd1-9b83-419b-b079-0d0bfafe7fe3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:58.601809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7abcd1-9b83-419b-b079-0d0bfafe7fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:58.601926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7abcd1-9b83-419b-b079-0d0bfafe7fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:58.621688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e7abcd1-9b83-419b-b079-0d0bfafe7fe3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:01:58.622892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e7abcd1-9b83-419b-b079-0d0bfafe7fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:01:58.631870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:01:58.677384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe74f-5ae0-4adc-aef3-c1aabb980e42 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:01:58.694448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77fe74f-5ae0-4adc-aef3-c1aabb980e42 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:01:58.700294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:01:58.701010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:01:58.701309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:01:58.701603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:01:58.711021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:01:58.712697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241cac09-bca5-4f36-92be-9e81c4779b74 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:01:58.722979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:02:00.778481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab31fa4-f481-498f-a05e-94694b0d2326 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:00.779192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab31fa4-f481-498f-a05e-94694b0d2326 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:00.779192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab31fa4-f481-498f-a05e-94694b0d2326 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:00.780457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1395] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:00.785993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed30f8-83c9-46c3-94ac-0a47ccffb021 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:00.786535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed30f8-83c9-46c3-94ac-0a47ccffb021 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:00.786535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed30f8-83c9-46c3-94ac-0a47ccffb021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:00.804692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfed30f8-83c9-46c3-94ac-0a47ccffb021 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:00.805913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfed30f8-83c9-46c3-94ac-0a47ccffb021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:00.815458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:00.858902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab31fa4-f481-498f-a05e-94694b0d2326 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:00.874113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab31fa4-f481-498f-a05e-94694b0d2326 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:00.880234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:00.880914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:00.881205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:00.881479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:00.892460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:00.894211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2b106-3b91-4737-b9f6-38da82f075f5 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:00.904335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:01.467516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26079ee-add4-4dcb-871d-b7823365ae89 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:01.467764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26079ee-add4-4dcb-871d-b7823365ae89 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:01.468046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26079ee-add4-4dcb-871d-b7823365ae89 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:01.469515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1398] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:01.475037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10337707-7e0f-4883-bd4e-5c43e9b12a38 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:01.475324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10337707-7e0f-4883-bd4e-5c43e9b12a38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:01.475599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10337707-7e0f-4883-bd4e-5c43e9b12a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:01.494076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10337707-7e0f-4883-bd4e-5c43e9b12a38 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:01.495302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10337707-7e0f-4883-bd4e-5c43e9b12a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:01.504217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:01.559418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085be73-03c5-4d59-a5b0-0f7dc14543e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:01.559707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085be73-03c5-4d59-a5b0-0f7dc14543e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:01.559985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085be73-03c5-4d59-a5b0-0f7dc14543e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:01.561477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1400] 10.222.0.16 () {54 vars in 962 bytes} [Tue Jun 21 03:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:01.566866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81344ac6-4b77-4f94-b29f-c8867f3e7a88 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:01.567161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81344ac6-4b77-4f94-b29f-c8867f3e7a88 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:01.567442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81344ac6-4b77-4f94-b29f-c8867f3e7a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:01.617449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-81344ac6-4b77-4f94-b29f-c8867f3e7a88 None None] MFA Rules not processed for user `9bdf485ec3ca4f3785bb47f3706c4466`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:01.620322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81344ac6-4b77-4f94-b29f-c8867f3e7a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:01.682394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Jun 21 03:02:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:01.690156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085be73-03c5-4d59-a5b0-0f7dc14543e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:01.701732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085be73-03c5-4d59-a5b0-0f7dc14543e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:01.705579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:01.706279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:01.706575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:01.706835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:01.716292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:01.717857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96535c-9118-445a-8e1c-6250b78f5710 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:01.728042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:02.452226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9339-23ba-4cc5-b475-208d33164355 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.452754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9339-23ba-4cc5-b475-208d33164355 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.452870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9339-23ba-4cc5-b475-208d33164355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.502262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e8e9339-23ba-4cc5-b475-208d33164355 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:02.503594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e8e9339-23ba-4cc5-b475-208d33164355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:02.546828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:02:02.572127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:02:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:02.616175 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f203-0c44-4018-ba65-5dc094cff8a4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.616704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f203-0c44-4018-ba65-5dc094cff8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.616704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f203-0c44-4018-ba65-5dc094cff8a4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.618042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1404] 10.222.0.16 () {54 vars in 942 bytes} [Tue Jun 21 03:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:02.625112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21def2b-f6fb-465b-b346-b86abd7f779c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.625405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21def2b-f6fb-465b-b346-b86abd7f779c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.625689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21def2b-f6fb-465b-b346-b86abd7f779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.676105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b21def2b-f6fb-465b-b346-b86abd7f779c None None] MFA Rules not processed for user `5b4f45c15c2949f5bd0e606b3cf00493`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:02.677448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b21def2b-f6fb-465b-b346-b86abd7f779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:02.755200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Tue Jun 21 03:02:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:02:02.764894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27f203-0c44-4018-ba65-5dc094cff8a4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:02.779513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27f203-0c44-4018-ba65-5dc094cff8a4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:02.784948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:02.785588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.785876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.786157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.794759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:02.796385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c44850-8bc8-41a5-b048-b324fdf928bf service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:02.805457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:02:02.945843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782efafa-223a-4843-8ad1-079349319323 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.946187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782efafa-223a-4843-8ad1-079349319323 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.946428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782efafa-223a-4843-8ad1-079349319323 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.947819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1407] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:02.954326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea9416-dd02-4359-ab97-adca206c273e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:02.954611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea9416-dd02-4359-ab97-adca206c273e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:02.954892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea9416-dd02-4359-ab97-adca206c273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:02.974068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-31ea9416-dd02-4359-ab97-adca206c273e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:02.975450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31ea9416-dd02-4359-ab97-adca206c273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:02.983633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:03.028572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782efafa-223a-4843-8ad1-079349319323 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:03.045085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782efafa-223a-4843-8ad1-079349319323 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:03.050547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:03.051204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:03.051489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:03.051795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:03.060695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:03.062355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf1d4-ee6c-49b9-bbee-9963862acbbc service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:03.072174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:12.968714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=69 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:02:13.067860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=59 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:02:25.046028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583021e-a4fc-40af-91e4-d93c1a67a2cd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:25.046670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583021e-a4fc-40af-91e4-d93c1a67a2cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:25.046670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583021e-a4fc-40af-91e4-d93c1a67a2cd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:25.047974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1410] 10.222.0.16 () {54 vars in 918 bytes} [Tue Jun 21 03:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:25.054190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae8ab-4443-4ab5-907b-9f754e637bd1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:25.054696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae8ab-4443-4ab5-907b-9f754e637bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:25.054696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae8ab-4443-4ab5-907b-9f754e637bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:25.083178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-420ae8ab-4443-4ab5-907b-9f754e637bd1 None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:25.084424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-420ae8ab-4443-4ab5-907b-9f754e637bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:25.094603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Jun 21 03:02:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:25.111242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30870-cdd8-48b9-ba34-5c05b9f2c57e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:25.112104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30870-cdd8-48b9-ba34-5c05b9f2c57e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:25.112401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30870-cdd8-48b9-ba34-5c05b9f2c57e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:25.113817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1412] 10.222.0.16 () {54 vars in 918 bytes} [Tue Jun 21 03:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:25.118669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58c3c-7930-4e5c-b135-fa9f9e0f2e15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:25.118963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58c3c-7930-4e5c-b135-fa9f9e0f2e15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:25.119234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d58c3c-7930-4e5c-b135-fa9f9e0f2e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:25.139648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-89d58c3c-7930-4e5c-b135-fa9f9e0f2e15 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:25.140870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89d58c3c-7930-4e5c-b135-fa9f9e0f2e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:25.152050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Tue Jun 21 03:02:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:27.629920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab18af-4d03-480f-b5b5-b3d056aade63 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:27.630803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab18af-4d03-480f-b5b5-b3d056aade63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:27.630803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab18af-4d03-480f-b5b5-b3d056aade63 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:27.633237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:27.638778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aeb63b-a561-4617-92d4-88a589aa7b7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:27.639247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aeb63b-a561-4617-92d4-88a589aa7b7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:27.639359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aeb63b-a561-4617-92d4-88a589aa7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:27.659297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-38aeb63b-a561-4617-92d4-88a589aa7b7d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:27.660522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-38aeb63b-a561-4617-92d4-88a589aa7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:27.668905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:27.712500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab18af-4d03-480f-b5b5-b3d056aade63 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:27.728758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab18af-4d03-480f-b5b5-b3d056aade63 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:27.734635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:27.735270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:27.735555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:27.735856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:27.744244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:27.745868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61917520-8eea-48c8-92cb-696305911cc1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:27.755741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:28.643176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d40714-8993-41f9-b610-c9ebb56f7374 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:28.644096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d40714-8993-41f9-b610-c9ebb56f7374 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:28.644096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d40714-8993-41f9-b610-c9ebb56f7374 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:28.646337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:28.652297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-aeeb-42d1-91c9-fc0a6f1db24e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:28.653083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-aeeb-42d1-91c9-fc0a6f1db24e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:28.653263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-aeeb-42d1-91c9-fc0a6f1db24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:28.673881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ded5bb41-aeeb-42d1-91c9-fc0a6f1db24e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:28.675084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ded5bb41-aeeb-42d1-91c9-fc0a6f1db24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:28.683648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:28.830994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d40714-8993-41f9-b610-c9ebb56f7374 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:28.857666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d40714-8993-41f9-b610-c9ebb56f7374 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:28.865504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:28.866175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:28.866460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:28.866737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:28.876205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:28.877932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886bf69-39d8-4f7f-a83b-cf8549473ad0 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:28.887827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:29.760704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb28b-b239-4ece-9c5d-4b38fd29fd79 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:29.761270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb28b-b239-4ece-9c5d-4b38fd29fd79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:29.761270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb28b-b239-4ece-9c5d-4b38fd29fd79 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:29.762658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:29.768846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b161ea-d5ba-422e-b04b-057cc62cea2b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:29.769638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b161ea-d5ba-422e-b04b-057cc62cea2b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:29.769741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b161ea-d5ba-422e-b04b-057cc62cea2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:29.799828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-17b161ea-d5ba-422e-b04b-057cc62cea2b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:29.801121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17b161ea-d5ba-422e-b04b-057cc62cea2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:29.810092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:29.960466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cb28b-b239-4ece-9c5d-4b38fd29fd79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:29.980024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cb28b-b239-4ece-9c5d-4b38fd29fd79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:29.987453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:29.988138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:29.988486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:29.988769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:29.997532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:29.999138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be809d-8b30-40fe-9e0a-70e57a78adcd service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:30.009797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:30.888156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a01ffe-28bd-4ef1-8534-20d0a7bdc2b9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:30.889120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a01ffe-28bd-4ef1-8534-20d0a7bdc2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:30.889548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a01ffe-28bd-4ef1-8534-20d0a7bdc2b9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:30.891871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:30.898058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffdf7f-0079-47d1-9317-f61b50cf797e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:30.898496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffdf7f-0079-47d1-9317-f61b50cf797e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:30.898806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffdf7f-0079-47d1-9317-f61b50cf797e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:30.921703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3ffdf7f-0079-47d1-9317-f61b50cf797e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:30.922916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3ffdf7f-0079-47d1-9317-f61b50cf797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:30.931729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:31.088466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a01ffe-28bd-4ef1-8534-20d0a7bdc2b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:31.117811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a01ffe-28bd-4ef1-8534-20d0a7bdc2b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:31.124007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:31.125064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:31.125352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:31.125631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:31.135031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:31.136680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bcd16-e2b5-480b-a34f-e587efe58084 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:31.146537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:32.020743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04211dc5-6f64-431c-b70a-05d5ad8d3a3c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:32.021656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04211dc5-6f64-431c-b70a-05d5ad8d3a3c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:32.021656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04211dc5-6f64-431c-b70a-05d5ad8d3a3c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:32.023808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:32.029335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a1f8b-5eee-4ddb-befd-56c9313c20d7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:32.029782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a1f8b-5eee-4ddb-befd-56c9313c20d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:32.029887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a1f8b-5eee-4ddb-befd-56c9313c20d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:32.050620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff3a1f8b-5eee-4ddb-befd-56c9313c20d7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:32.051823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff3a1f8b-5eee-4ddb-befd-56c9313c20d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:32.060260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:02:32.210007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04211dc5-6f64-431c-b70a-05d5ad8d3a3c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:32.226041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04211dc5-6f64-431c-b70a-05d5ad8d3a3c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:32.231677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:32.232313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:32.232623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:32.232902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:32.241421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:32.243020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14883563-d3c6-47e4-b5fe-b2e9ff0770da service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:32.252851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:33.062915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b0ede-d544-42d2-81d7-b9a9adb7167a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:33.063535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b0ede-d544-42d2-81d7-b9a9adb7167a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:33.063535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b0ede-d544-42d2-81d7-b9a9adb7167a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:33.065554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:33.072970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4ceb9-8042-44e1-b7fc-85ebe94ea0c9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:33.073388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4ceb9-8042-44e1-b7fc-85ebe94ea0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:33.073547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4ceb9-8042-44e1-b7fc-85ebe94ea0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:33.094946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20c4ceb9-8042-44e1-b7fc-85ebe94ea0c9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:33.096240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20c4ceb9-8042-44e1-b7fc-85ebe94ea0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:33.105441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:33.255580 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b0ede-d544-42d2-81d7-b9a9adb7167a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:33.272722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b0ede-d544-42d2-81d7-b9a9adb7167a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:33.279257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:33.279903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:33.280182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:33.280444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:33.289514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:33.291114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf78d3b-f4b8-4173-93b6-b0583f6ff4f4 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:33.301080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:34.097146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d319-acc2-45f4-a235-91d92042130e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:34.097728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d319-acc2-45f4-a235-91d92042130e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:34.097812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d319-acc2-45f4-a235-91d92042130e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:34.099813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:34.105991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517790-6dc0-45c7-91e9-310eb7356989 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:34.106502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517790-6dc0-45c7-91e9-310eb7356989 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:34.106599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a517790-6dc0-45c7-91e9-310eb7356989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:34.127139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5a517790-6dc0-45c7-91e9-310eb7356989 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:34.128650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a517790-6dc0-45c7-91e9-310eb7356989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:34.136772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:34.296379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d319-acc2-45f4-a235-91d92042130e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:34.312830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d319-acc2-45f4-a235-91d92042130e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:34.318819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:34.319447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:34.319728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:34.319999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:34.328297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:34.329900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683346a-2823-490f-9a79-9a4f6bb6139d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:34.339132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:35.146283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cb367-ef53-4bc4-8e80-67030bc03dd2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:35.146893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cb367-ef53-4bc4-8e80-67030bc03dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:35.146893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cb367-ef53-4bc4-8e80-67030bc03dd2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:35.148958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:35.157756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb864a-84e1-46f8-a893-747d133de8a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:35.158212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb864a-84e1-46f8-a893-747d133de8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:35.158451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb864a-84e1-46f8-a893-747d133de8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:35.178122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8bdb864a-84e1-46f8-a893-747d133de8a0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:35.179331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8bdb864a-84e1-46f8-a893-747d133de8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:35.189049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:35.340460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cb367-ef53-4bc4-8e80-67030bc03dd2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:35.356269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cb367-ef53-4bc4-8e80-67030bc03dd2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:35.362415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:35.363056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:35.363336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:35.363604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:35.372193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:35.373822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0296155-334a-443f-94c3-885629cf4d61 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:35.383778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:36.181156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de607988-1e56-46f4-9d83-1a36a7ea57d4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:36.181671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de607988-1e56-46f4-9d83-1a36a7ea57d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:36.181671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de607988-1e56-46f4-9d83-1a36a7ea57d4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:36.183893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:02:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:36.189190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c348-fc5d-44be-a3b4-214f63bbc2f4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:36.189722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c348-fc5d-44be-a3b4-214f63bbc2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:36.189947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270c348-fc5d-44be-a3b4-214f63bbc2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:36.208302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c270c348-fc5d-44be-a3b4-214f63bbc2f4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:36.209531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c270c348-fc5d-44be-a3b4-214f63bbc2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:36.218940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:36.368226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de607988-1e56-46f4-9d83-1a36a7ea57d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:36.391748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de607988-1e56-46f4-9d83-1a36a7ea57d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:36.398755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:36.399407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:36.399704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:36.399969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:36.408787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:36.410366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cabe58-f2dc-405d-b687-2465164e4fe3 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:36.420330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:38.338571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a708c-97f8-4128-b4e8-5c19ad63b7ae None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:38.339190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a708c-97f8-4128-b4e8-5c19ad63b7ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:38.339190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a708c-97f8-4128-b4e8-5c19ad63b7ae None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:38.340606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:38.346288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aef4e-1735-46c0-a7f7-64a6c3ebbf5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:38.346735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aef4e-1735-46c0-a7f7-64a6c3ebbf5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:38.346844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aef4e-1735-46c0-a7f7-64a6c3ebbf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:38.365310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f3aef4e-1735-46c0-a7f7-64a6c3ebbf5b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:38.366515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f3aef4e-1735-46c0-a7f7-64a6c3ebbf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:38.377629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:38.416978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a708c-97f8-4128-b4e8-5c19ad63b7ae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:38.436464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a708c-97f8-4128-b4e8-5c19ad63b7ae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:38.441986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:38.442633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:38.442920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:38.443197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:38.452959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:38.454409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7dbb9-439d-495b-a517-0f936e5eb79e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:38.464287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:40.635687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293cdd-96b4-40c1-849c-3634c06b2085 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:40.636291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293cdd-96b4-40c1-849c-3634c06b2085 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:40.636291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70293cdd-96b4-40c1-849c-3634c06b2085 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:40.637672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:40.644115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7a1c-1c0e-4473-a67a-464f1f6be173 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:40.644593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7a1c-1c0e-4473-a67a-464f1f6be173 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:40.644711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7a1c-1c0e-4473-a67a-464f1f6be173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:40.668898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d97d7a1c-1c0e-4473-a67a-464f1f6be173 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:40.670095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d97d7a1c-1c0e-4473-a67a-464f1f6be173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:40.679905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:40.718973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70293cdd-96b4-40c1-849c-3634c06b2085 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:40.731518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70293cdd-96b4-40c1-849c-3634c06b2085 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:40.735612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:40.736258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:40.736661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:40.736901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:40.746240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:40.747656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164dd2b-9fcf-437b-9881-510e14ceb39d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:40.757983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:43.196906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d934-5847-4b59-9984-a894c5515ad0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:43.197435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d934-5847-4b59-9984-a894c5515ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:43.197435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d934-5847-4b59-9984-a894c5515ad0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:43.198891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:43.204514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628690ca-3d2b-4f53-83ec-da4b3ff6bb3d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:43.204944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628690ca-3d2b-4f53-83ec-da4b3ff6bb3d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:43.205212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628690ca-3d2b-4f53-83ec-da4b3ff6bb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:43.231860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-628690ca-3d2b-4f53-83ec-da4b3ff6bb3d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:43.233462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-628690ca-3d2b-4f53-83ec-da4b3ff6bb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:43.244248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:43.284751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d934-5847-4b59-9984-a894c5515ad0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:43.300995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d934-5847-4b59-9984-a894c5515ad0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:43.304799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:43.305445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:43.305730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:43.306003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:43.316059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:43.317581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338305f-bf27-4201-9a80-683a9717e070 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:43.327244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:45.428955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50ff-6a9f-4045-82dd-8c546255f9cd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:45.429503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50ff-6a9f-4045-82dd-8c546255f9cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:45.429503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50ff-6a9f-4045-82dd-8c546255f9cd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:45.430833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:45.437266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f07f5-1e7c-41c6-b1d3-fe64f9a742f2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:45.438001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f07f5-1e7c-41c6-b1d3-fe64f9a742f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:45.438181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f07f5-1e7c-41c6-b1d3-fe64f9a742f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:45.464071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f66f07f5-1e7c-41c6-b1d3-fe64f9a742f2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:45.465301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f66f07f5-1e7c-41c6-b1d3-fe64f9a742f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:45.474649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:45.514924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e50ff-6a9f-4045-82dd-8c546255f9cd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:45.528042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e50ff-6a9f-4045-82dd-8c546255f9cd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:45.532118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:45.532804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:45.533116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:45.533394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:45.542693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:45.544110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f632b2-f72b-4111-931b-a126cfcefecc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:45.553876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:47.684916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9256c5-dc1f-4f9e-a118-2907c59dddd5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:47.685801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9256c5-dc1f-4f9e-a118-2907c59dddd5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:47.685801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9256c5-dc1f-4f9e-a118-2907c59dddd5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:47.688037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:47.693733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad57c8b-29b6-4063-aa1a-b62976232713 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:47.694188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad57c8b-29b6-4063-aa1a-b62976232713 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:47.694292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad57c8b-29b6-4063-aa1a-b62976232713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:47.714762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3ad57c8b-29b6-4063-aa1a-b62976232713 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:47.716882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ad57c8b-29b6-4063-aa1a-b62976232713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:47.727551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:47.765854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9256c5-dc1f-4f9e-a118-2907c59dddd5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:47.778304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9256c5-dc1f-4f9e-a118-2907c59dddd5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:47.782267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:47.782912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:47.783196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:47.783495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:47.792822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:47.794269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8ace2-f236-4f89-a211-b48076491712 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:47.803760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:49.882775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b0f5-b883-4e69-95e9-74e02c48581e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:49.883358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b0f5-b883-4e69-95e9-74e02c48581e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:49.883358 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802b0f5-b883-4e69-95e9-74e02c48581e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:49.884692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:49.891385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4717-bf86-460c-9b13-77058b1457cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:49.892114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4717-bf86-460c-9b13-77058b1457cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:49.892384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4717-bf86-460c-9b13-77058b1457cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:49.915288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-154a4717-bf86-460c-9b13-77058b1457cd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:49.916500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-154a4717-bf86-460c-9b13-77058b1457cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:49.925670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:02:49.966815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802b0f5-b883-4e69-95e9-74e02c48581e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:49.979834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802b0f5-b883-4e69-95e9-74e02c48581e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:49.984404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:49.985077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:49.985362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:49.985635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:49.995286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:49.996784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c340e-3316-472e-b885-104cdf62afcf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:50.007309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:52.077054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c69b9-cfc8-48a3-9096-7c0f285067ad None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:52.077679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c69b9-cfc8-48a3-9096-7c0f285067ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:52.077679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c69b9-cfc8-48a3-9096-7c0f285067ad None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:52.078971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:52.085285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24b82-d2db-44b2-9d8e-b7e6008d4978 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:52.085750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24b82-d2db-44b2-9d8e-b7e6008d4978 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:52.085895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24b82-d2db-44b2-9d8e-b7e6008d4978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:52.115131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-21e24b82-d2db-44b2-9d8e-b7e6008d4978 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:52.116346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21e24b82-d2db-44b2-9d8e-b7e6008d4978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:52.125983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:52.163445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c69b9-cfc8-48a3-9096-7c0f285067ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:52.175598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c69b9-cfc8-48a3-9096-7c0f285067ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:52.179458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:52.180106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:52.180388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:52.180692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:52.190378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:52.191790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b48bd9-3915-41a5-a681-47e6fa6989ea service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:52.202520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:54.245847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8487-c0fd-4f7d-8175-ae602f928e4b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:54.247072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8487-c0fd-4f7d-8175-ae602f928e4b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:54.247357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d8487-c0fd-4f7d-8175-ae602f928e4b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:54.248825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:54.257615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fcf04-99ff-4349-a514-bb27e2c6c77e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:54.258098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fcf04-99ff-4349-a514-bb27e2c6c77e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:54.258204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fcf04-99ff-4349-a514-bb27e2c6c77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:54.265062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=79 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:02:54.277597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc5fcf04-99ff-4349-a514-bb27e2c6c77e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:54.278846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc5fcf04-99ff-4349-a514-bb27e2c6c77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:54.287942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:02:54.329606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d8487-c0fd-4f7d-8175-ae602f928e4b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:54.335764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=73 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:02:54.343738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d8487-c0fd-4f7d-8175-ae602f928e4b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:54.348037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:54.348728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:54.349015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:54.349308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:54.359325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:54.360817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96f871-744f-4a2c-8b9d-fc85c0533eaf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:54.370508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:56.427039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ddfae-2e0a-44fd-b132-8396274ed328 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:56.427799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ddfae-2e0a-44fd-b132-8396274ed328 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:56.427799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ddfae-2e0a-44fd-b132-8396274ed328 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:56.430048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:56.436167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c89d1-9d99-4a2f-99eb-8685434539b8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:56.436719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c89d1-9d99-4a2f-99eb-8685434539b8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:56.436886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c89d1-9d99-4a2f-99eb-8685434539b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:56.458074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-995c89d1-9d99-4a2f-99eb-8685434539b8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:56.459318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-995c89d1-9d99-4a2f-99eb-8685434539b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:56.468389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:56.506640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ddfae-2e0a-44fd-b132-8396274ed328 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:56.520011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ddfae-2e0a-44fd-b132-8396274ed328 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:56.524320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:56.524998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:56.525284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:56.525561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:56.535737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:56.537218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ed889-3bc1-4647-bea6-6d529697418e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:56.546897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:02:58.549519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fc94-0bb6-4301-b525-d20b1455b40c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:58.550418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fc94-0bb6-4301-b525-d20b1455b40c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:58.550418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717fc94-0bb6-4301-b525-d20b1455b40c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:58.552880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:02:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:02:58.558343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4522d-2f5e-4501-a77d-a71afe7360d3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:58.558794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4522d-2f5e-4501-a77d-a71afe7360d3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:58.558899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4522d-2f5e-4501-a77d-a71afe7360d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:58.577545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-09a4522d-2f5e-4501-a77d-a71afe7360d3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:02:58.578731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09a4522d-2f5e-4501-a77d-a71afe7360d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:02:58.587115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:02:58.626963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717fc94-0bb6-4301-b525-d20b1455b40c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:02:58.642143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717fc94-0bb6-4301-b525-d20b1455b40c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:02:58.648678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:02:58.649580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:02:58.649930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:02:58.650315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:02:58.662810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:02:58.664247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63d6bd-b911-406d-801f-9a3131da75fa service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:02:58.674570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:08.572299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:03:08.669547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=14 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:03:18.420401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20612aa-d37a-4e5a-af35-aa696090773d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:18.421009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20612aa-d37a-4e5a-af35-aa696090773d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:18.421009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20612aa-d37a-4e5a-af35-aa696090773d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:18.422322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1471] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:18.429171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600e30-f723-4b39-ba30-fb051f16a2dd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:18.429707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600e30-f723-4b39-ba30-fb051f16a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:18.429707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07600e30-f723-4b39-ba30-fb051f16a2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:18.456467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-07600e30-f723-4b39-ba30-fb051f16a2dd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:18.457757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07600e30-f723-4b39-ba30-fb051f16a2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:18.466997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:18.519623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc81fe-87b2-42b3-a5b6-a914d89ebbb7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:18.519918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc81fe-87b2-42b3-a5b6-a914d89ebbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:18.520200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc81fe-87b2-42b3-a5b6-a914d89ebbb7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:18.521654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1473] 10.222.0.16 () {54 vars in 962 bytes} [Tue Jun 21 03:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:18.526516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fbde-a940-4ae1-b89c-db95fb3dbf73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:18.526829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fbde-a940-4ae1-b89c-db95fb3dbf73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:18.527091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6fbde-a940-4ae1-b89c-db95fb3dbf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:18.545913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-38a6fbde-a940-4ae1-b89c-db95fb3dbf73 None None] MFA Rules not processed for user `9bdf485ec3ca4f3785bb47f3706c4466`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:18.547137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38a6fbde-a940-4ae1-b89c-db95fb3dbf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:18.555156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Tue Jun 21 03:03:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:18.562858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc81fe-87b2-42b3-a5b6-a914d89ebbb7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:18.575273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc81fe-87b2-42b3-a5b6-a914d89ebbb7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:18.579374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:18.580034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:18.580322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:18.580630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:18.590045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:18.591502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a3490-31f9-4d97-b8cc-71aab1c2bb2e service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:18.601166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:20.886367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a81d-28c5-442f-ac8d-b492e59280df None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:20.886973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a81d-28c5-442f-ac8d-b492e59280df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:20.886973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a81d-28c5-442f-ac8d-b492e59280df None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:20.888324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1476] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:20.895044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b574d-d357-47ea-a356-d62252555b2f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:20.895515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b574d-d357-47ea-a356-d62252555b2f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:20.895619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b574d-d357-47ea-a356-d62252555b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:20.918945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a39b574d-d357-47ea-a356-d62252555b2f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:20.920181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a39b574d-d357-47ea-a356-d62252555b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:20.929388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:20.953659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a81d-28c5-442f-ac8d-b492e59280df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:20.971293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a81d-28c5-442f-ac8d-b492e59280df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:20.976716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:20.977377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:20.977656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:20.977927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:20.978659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:20.980506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:03:20.980506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:03:20.980506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:03:20.981671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb6e50-7f8b-4234-8acf-7e2586a8b89e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:21.001788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Tue Jun 21 03:03:20 2022] GET /identity/v3/projects => generated 2553 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:03:23.063955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9b62-6d88-490e-9eff-551d15260fe2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:23.064592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9b62-6d88-490e-9eff-551d15260fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:23.064592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9b62-6d88-490e-9eff-551d15260fe2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:23.065900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1479] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:23.072258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ef35-f286-45ad-a280-948c17552d5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:23.072735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ef35-f286-45ad-a280-948c17552d5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:23.072841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ef35-f286-45ad-a280-948c17552d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:23.099737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dad0ef35-f286-45ad-a280-948c17552d5d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:23.100979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dad0ef35-f286-45ad-a280-948c17552d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:23.110024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:23.152980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff9b62-6d88-490e-9eff-551d15260fe2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:23.168510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff9b62-6d88-490e-9eff-551d15260fe2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:23.177464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:23.178123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:23.178407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:23.178678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:23.187349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:23.189038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f67c43-affc-46a6-805e-2755badf3669 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:23.198390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:25.471742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2854-a2ae-4bf4-aee1-f86b03a3184e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:25.472658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2854-a2ae-4bf4-aee1-f86b03a3184e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:25.472658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf2854-a2ae-4bf4-aee1-f86b03a3184e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:25.474873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1482] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:25.482838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a95bf-fb55-45fc-ae54-1f71a59a71aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:25.483319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a95bf-fb55-45fc-ae54-1f71a59a71aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:25.483449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a95bf-fb55-45fc-ae54-1f71a59a71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:25.504753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-962a95bf-fb55-45fc-ae54-1f71a59a71aa None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:25.505956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-962a95bf-fb55-45fc-ae54-1f71a59a71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:25.514546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:03:25.551560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2854-a2ae-4bf4-aee1-f86b03a3184e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:25.564894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf2854-a2ae-4bf4-aee1-f86b03a3184e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:25.569432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:25.570074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:25.570354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:25.570629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:25.581182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:25.582616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e89b8-9a80-42f2-82f2-d3e5e19c9cd0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:25.592988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:27.660383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01708-bc66-449f-ab42-1061a91dcb1c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:27.661259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01708-bc66-449f-ab42-1061a91dcb1c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:27.661404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01708-bc66-449f-ab42-1061a91dcb1c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:27.663701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1485] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:27.669288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c48a-2ff9-4218-964a-2cd8f2d21725 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:27.669742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c48a-2ff9-4218-964a-2cd8f2d21725 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:27.669858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4c48a-2ff9-4218-964a-2cd8f2d21725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:27.694255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b9a4c48a-2ff9-4218-964a-2cd8f2d21725 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:27.695464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9a4c48a-2ff9-4218-964a-2cd8f2d21725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:27.704888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:27.742609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e01708-bc66-449f-ab42-1061a91dcb1c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:27.754800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e01708-bc66-449f-ab42-1061a91dcb1c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:27.758712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:27.759355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:27.759640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:27.759924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:27.769554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:27.771067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747414fc-a96d-4b7e-abce-b7559df885f3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:27.780832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:29.940768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4a5-5dfb-43ab-bc6b-e635ebbc31fa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:29.941680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4a5-5dfb-43ab-bc6b-e635ebbc31fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:29.941680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e4a5-5dfb-43ab-bc6b-e635ebbc31fa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:29.943799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1488] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:29.950394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259408-5c5b-4eab-a0f1-08e4a7a97392 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:29.951180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259408-5c5b-4eab-a0f1-08e4a7a97392 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:29.951397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259408-5c5b-4eab-a0f1-08e4a7a97392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:29.976836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1259408-5c5b-4eab-a0f1-08e4a7a97392 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:29.978049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1259408-5c5b-4eab-a0f1-08e4a7a97392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:29.986827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:30.027256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e4a5-5dfb-43ab-bc6b-e635ebbc31fa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:30.041019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e4a5-5dfb-43ab-bc6b-e635ebbc31fa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:30.045310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:30.045975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:30.046255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:30.046518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:30.056029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:30.057500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b910f-2497-47b3-aa1b-d38b5b113fac service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:30.067049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:32.261219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2292b-896b-4833-9073-acd2b81a72db None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:32.261781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2292b-896b-4833-9073-acd2b81a72db None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:32.261781 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2292b-896b-4833-9073-acd2b81a72db None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:32.263198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1491] 10.222.0.16 () {54 vars in 926 bytes} [Tue Jun 21 03:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:03:32.269721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda900b-f14f-461c-9247-f5beee0ae987 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:32.270496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda900b-f14f-461c-9247-f5beee0ae987 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:32.270622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda900b-f14f-461c-9247-f5beee0ae987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:32.298012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eeda900b-f14f-461c-9247-f5beee0ae987 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:03:32.299673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eeda900b-f14f-461c-9247-f5beee0ae987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:03:32.311971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Tue Jun 21 03:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:03:32.353419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2292b-896b-4833-9073-acd2b81a72db None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:03:32.373154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2292b-896b-4833-9073-acd2b81a72db None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:03:32.379019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:03:32.379662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:03:32.379942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:03:32.380218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:03:32.390521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:03:32.392006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2f373-9eaa-4f6e-add3-64eef5e4b6c9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:03:32.401996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:03:42.289956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=33 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:03:42.397592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=33 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:05:04.336617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.337503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.337503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.359778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:05:04.361148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:04.370126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1494] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:05:04.379795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.380357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.380357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.399548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:05:04.400783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:04.411163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1495] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:05:04.416500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.432081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb8d0f-f181-4dbc-b1d3-a28cdf118971 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.439864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.440569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.440855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.441125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.441861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:05:04.442174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.443215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.443215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:05:04.443215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:05:04.444102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.456627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:05:04.464848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.474365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575179f-09d3-4cbb-8749-41bd9e0fc331 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.480005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.480719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.480998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.481266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.481968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.483778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.483778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:05:04.483778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:05:04.485163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.525951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:05:04.532658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.543576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457bdf4-2f47-4fe4-97f1-4b7f7d9b0030 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.602924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.603515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.603800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.604076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.604807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.605843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.605843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:05:04.605843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:05:04.607055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.631732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:05:04.636657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.648998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f7d11-199c-40d0-b1e8-3f86649bf802 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.653988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.654647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.654955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.655217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.655950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.657049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.657049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:05:04.657049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:05:04.657989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.670820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:05:04.675256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.687506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1a7a3-0ecb-4515-8827-fe4d2d6fbf7d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.692587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.693259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.693544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.693820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.694554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.695574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.695574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:05:04.695574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:05:04.696495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f985-8299-4a5d-bba1-afd8e2fbefd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.711134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:05:04.716345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:04.733814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22b29d-6405-47e6-8707-a2b376072c87 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:04.740312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:04.741252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.741542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.741820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] PATH_INFO: `/v3/projects/cc276650f57d4c25a094f8f3ff4054f4/users/6c097a078fe64df79ecc53b0dc78be32/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.757474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc276650f57d4c25a094f8f3ff4054f4, user_id=6c097a078fe64df79ecc53b0dc78be32, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:04.758528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:04.758528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:05:04.758528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:05:04.760448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8cf02-9c1c-4695-8cf5-c5f5908d9ac8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:04.775271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:05:04 2022] PUT /identity/v3/projects/cc276650f57d4c25a094f8f3ff4054f4/users/6c097a078fe64df79ecc53b0dc78be32/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:05:04.787128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba473d96-be00-4c6c-a13f-fdeecd7c9865 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.787422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba473d96-be00-4c6c-a13f-fdeecd7c9865 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.787705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba473d96-be00-4c6c-a13f-fdeecd7c9865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.815844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba473d96-be00-4c6c-a13f-fdeecd7c9865 None None] MFA Rules not processed for user `6c097a078fe64df79ecc53b0dc78be32`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:05:04.817106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba473d96-be00-4c6c-a13f-fdeecd7c9865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:04.886642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1502] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:05:04.901982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c84d-838c-440f-97c3-f82ec90ee85b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.902272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c84d-838c-440f-97c3-f82ec90ee85b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.902533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c84d-838c-440f-97c3-f82ec90ee85b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.903920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1503] 10.222.0.16 () {54 vars in 963 bytes} [Tue Jun 21 03:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:05:04.909288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a301f-f9eb-4d15-a8cb-e89bf3cf2fff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:04.909588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a301f-f9eb-4d15-a8cb-e89bf3cf2fff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:04.909869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a301f-f9eb-4d15-a8cb-e89bf3cf2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:04.930011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-550a301f-f9eb-4d15-a8cb-e89bf3cf2fff None None] MFA Rules not processed for user `18bc17792b4d4879a1fd533206245694`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:05:04.931238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-550a301f-f9eb-4d15-a8cb-e89bf3cf2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:04.981521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:05:04.993164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Tue Jun 21 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:05:05.001702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832c84d-838c-440f-97c3-f82ec90ee85b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:05.013923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832c84d-838c-440f-97c3-f82ec90ee85b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:05.018624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:05.019264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.019550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.019833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.028897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:05.030407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:05.040447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:05:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:05:05.098906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37179cf-6273-48b3-9825-4ef40866ac16 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.099204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37179cf-6273-48b3-9825-4ef40866ac16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.099481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37179cf-6273-48b3-9825-4ef40866ac16 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.100832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1506] 10.222.0.16 () {52 vars in 829 bytes} [Tue Jun 21 03:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:05:05.104617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:05.171163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d24a-c7f8-4332-adb0-a401721e5aea service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:05.176719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:05.177610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.177895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.178172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] PATH_INFO: `/v3/users/6c097a078fe64df79ecc53b0dc78be32` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.179430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c097a078fe64df79ecc53b0dc78be32)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:05.180515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:05.180515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:05:05.180515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:05:05.181719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:05.260256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c097a078fe64df79ecc53b0dc78be32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.261079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c097a078fe64df79ecc53b0dc78be32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.279107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c097a078fe64df79ecc53b0dc78be32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.279808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c097a078fe64df79ecc53b0dc78be32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.292138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c097a078fe64df79ecc53b0dc78be32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.296134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:05:05.310542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:05:05 2022] DELETE /identity/v3/users/6c097a078fe64df79ecc53b0dc78be32 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:05:05.315759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.316083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.316368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.334939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:05:05.336185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:05.396432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1508] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:05:05.406160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:05.407788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:05.467737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77713df9-f23c-4f1c-bc06-d297918529b0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:05.471820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:05.472501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.472780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.473056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.482485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:05.483945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46528-aae5-4b3c-a2e1-c360728ce74b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:05.493682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:05:05.866936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:05:05.868591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:05:05.881257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726d7a5-0617-4aa6-affa-1f3cb05670be None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:05:05.886270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:05:05.887152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:05:05.887460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:05:05.887752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] PATH_INFO: `/v3/projects/cc276650f57d4c25a094f8f3ff4054f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:05:05.889029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc276650f57d4c25a094f8f3ff4054f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:05:05.890098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:05:05.890098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:05:05.890098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:05:05.891309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:05:05.957522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc276650f57d4c25a094f8f3ff4054f4', 'request_id': 'req-a16a3d63-061b-47f2-9658-feb052e54148'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.964586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc276650f57d4c25a094f8f3ff4054f4', 'request_id': 'req-a16a3d63-061b-47f2-9658-feb052e54148'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.976704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc276650f57d4c25a094f8f3ff4054f4', 'request_id': 'req-a16a3d63-061b-47f2-9658-feb052e54148'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.995315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] The token cache is being invalidate because project cc276650f57d4c25a094f8f3ff4054f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:05:05.995608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.996443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a16a3d63-061b-47f2-9658-feb052e54148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:05:05.998395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:05:05 2022] DELETE /identity/v3/projects/cc276650f57d4c25a094f8f3ff4054f4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:05:15.489377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:05:15.992093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:06:38.878850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:38.879531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:38.879531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:38.880744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:06:38.886077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abf449-5ca6-4b26-9dd0-0b710e8a4db2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:38.886546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abf449-5ca6-4b26-9dd0-0b710e8a4db2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:38.886649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abf449-5ca6-4b26-9dd0-0b710e8a4db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:38.905316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23abf449-5ca6-4b26-9dd0-0b710e8a4db2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:38.906534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23abf449-5ca6-4b26-9dd0-0b710e8a4db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:38.978947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:39.141482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:39.144993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:39.231678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e43ea2-f440-4d93-a016-b4d60634206b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:39.238178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:39.238831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:39.239114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:39.239388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:39.248235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:39.249921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e77ec-f89f-43f5-98ab-6b2ae58ed832 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:39.259474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:06:40.090497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f8c2c-571f-4670-9612-40e5b823a602 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:40.091422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f8c2c-571f-4670-9612-40e5b823a602 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:40.091422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f8c2c-571f-4670-9612-40e5b823a602 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:40.093800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:06:40.100045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d53a-820d-4831-a50e-291050a0d67d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:40.100512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d53a-820d-4831-a50e-291050a0d67d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:40.100695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2d53a-820d-4831-a50e-291050a0d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:40.128635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fdc2d53a-820d-4831-a50e-291050a0d67d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:40.129857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fdc2d53a-820d-4831-a50e-291050a0d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:40.138087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:40.294063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f8c2c-571f-4670-9612-40e5b823a602 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:40.310699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f8c2c-571f-4670-9612-40e5b823a602 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:40.317029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:40.317682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:40.317966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:40.318229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:40.327388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:40.329089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ccf9e-e793-45d3-aac0-3e4b0cd3328b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:40.339149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:06:41.105508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaaae9-5ef4-4cf8-9910-558a0231d858 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:41.106109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaaae9-5ef4-4cf8-9910-558a0231d858 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:41.106109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaaae9-5ef4-4cf8-9910-558a0231d858 None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:41.107386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1517] 10.222.0.36 () {54 vars in 925 bytes} [Tue Jun 21 03:06:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:06:41.114645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc9da-fd3e-48b3-b883-f65cbf6eb61b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:41.115088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc9da-fd3e-48b3-b883-f65cbf6eb61b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:41.115199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc9da-fd3e-48b3-b883-f65cbf6eb61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:41.135366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b2dc9da-fd3e-48b3-b883-f65cbf6eb61b None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:41.136714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b2dc9da-fd3e-48b3-b883-f65cbf6eb61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:41.203051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Jun 21 03:06:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:50.089771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092b1ab-8bbd-4c09-a5db-96cd8970137c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:50.090336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092b1ab-8bbd-4c09-a5db-96cd8970137c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:50.090336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092b1ab-8bbd-4c09-a5db-96cd8970137c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:50.092311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1519] 10.222.0.16 () {54 vars in 960 bytes} [Tue Jun 21 03:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:06:50.100991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfad8-96a7-4fa6-8ea8-7392bc61edec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:50.101458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfad8-96a7-4fa6-8ea8-7392bc61edec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:50.101563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bfad8-96a7-4fa6-8ea8-7392bc61edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:50.120989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=37 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:06:50.126392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9a1bfad8-96a7-4fa6-8ea8-7392bc61edec None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:50.127653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a1bfad8-96a7-4fa6-8ea8-7392bc61edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:50.137805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Tue Jun 21 03:06:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:50.147081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092b1ab-8bbd-4c09-a5db-96cd8970137c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:50.162501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092b1ab-8bbd-4c09-a5db-96cd8970137c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:50.168203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:50.168870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:50.169186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:50.169465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:50.178495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:50.180102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52416d34-1a75-4fe5-ac3a-ad086a110638 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:50.190169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:06:51.582505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c1e-adc5-4127-a7ec-afcccba5dfeb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:51.583369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c1e-adc5-4127-a7ec-afcccba5dfeb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:51.583769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4c1e-adc5-4127-a7ec-afcccba5dfeb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:51.586153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:06:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:06:51.592146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6e39-5f24-4add-8e1f-db4a4bc22824 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:51.592954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6e39-5f24-4add-8e1f-db4a4bc22824 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:51.593113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec6e39-5f24-4add-8e1f-db4a4bc22824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:51.619702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61ec6e39-5f24-4add-8e1f-db4a4bc22824 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:51.620944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61ec6e39-5f24-4add-8e1f-db4a4bc22824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:51.630843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:51.785919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d4c1e-adc5-4127-a7ec-afcccba5dfeb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:51.802510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d4c1e-adc5-4127-a7ec-afcccba5dfeb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:51.808551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:51.809194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:51.809481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:51.809766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:51.818726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:51.820321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82e999-5911-495f-95d0-30c007a503f9 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:51.830322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:06:52.642847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae20e-0893-4115-ae74-289128e755a1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:52.643466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae20e-0893-4115-ae74-289128e755a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:52.643466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ae20e-0893-4115-ae74-289128e755a1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:52.644834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 03:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:06:52.650707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7862-b6f7-4bcf-ac4e-dfc8c8f22f47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:52.651427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7862-b6f7-4bcf-ac4e-dfc8c8f22f47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:52.651583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7862-b6f7-4bcf-ac4e-dfc8c8f22f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:52.674677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3ec7862-b6f7-4bcf-ac4e-dfc8c8f22f47 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:52.675897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3ec7862-b6f7-4bcf-ac4e-dfc8c8f22f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:52.685489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 03:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:52.848879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ae20e-0893-4115-ae74-289128e755a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:52.874653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ae20e-0893-4115-ae74-289128e755a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:52.883582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:52.884644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:52.885111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:52.885558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:52.900264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:52.901960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8569c-72e3-4f8b-9299-cea6e64aa8ab service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:52.912132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:06:56.883821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:56.884396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:56.884396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:56.885701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:06:56.892661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441e99-8b53-4ebf-bb75-3d2ef087826b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:56.893119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441e99-8b53-4ebf-bb75-3d2ef087826b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:56.893253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441e99-8b53-4ebf-bb75-3d2ef087826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:56.914270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5d441e99-8b53-4ebf-bb75-3d2ef087826b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:56.915470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5d441e99-8b53-4ebf-bb75-3d2ef087826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:56.924279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:56.963400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:56.965277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:57.032701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ec3e7-571a-4355-8a38-c5eb2a600730 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:57.036979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:57.037618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:57.037901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:57.038176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:57.047745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:57.049204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b672e4-362d-4da8-9e6a-6f9d40639d1d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:57.054179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:06:57.059664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:06:59.042656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba623-d6b8-44c2-99b1-c914f2e582b8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:59.043637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba623-d6b8-44c2-99b1-c914f2e582b8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:59.043637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba623-d6b8-44c2-99b1-c914f2e582b8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:59.045843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:06:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:06:59.052941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e32d6-d34a-4214-afde-33f3277b3ecc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:59.053392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e32d6-d34a-4214-afde-33f3277b3ecc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:59.053516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e32d6-d34a-4214-afde-33f3277b3ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:59.073404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd1e32d6-d34a-4214-afde-33f3277b3ecc None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:06:59.074594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd1e32d6-d34a-4214-afde-33f3277b3ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:06:59.083053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:06:59.129299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ba623-d6b8-44c2-99b1-c914f2e582b8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:06:59.141920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ba623-d6b8-44c2-99b1-c914f2e582b8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:06:59.146488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:06:59.147179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:06:59.147476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:06:59.147750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:06:59.158560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:06:59.160044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65812afa-e2ef-41f9-b04e-7298c81cdeb4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:06:59.170138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:07:02.684286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:02.684875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:02.684875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:02.686301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:07:02.692938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10f09-ca50-424a-85ea-4f6a5bdc4ee3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:02.693717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10f09-ca50-424a-85ea-4f6a5bdc4ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:02.693889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e10f09-ca50-424a-85ea-4f6a5bdc4ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:02.721739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-89e10f09-ca50-424a-85ea-4f6a5bdc4ee3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:07:02.722936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89e10f09-ca50-424a-85ea-4f6a5bdc4ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:07:02.731225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:07:03.036693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:07:03.038519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:07:03.098278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e708b8e-b068-423e-b02d-5ff5b025aaa6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:07:03.102191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:07:03.102834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:03.103113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:03.103391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:03.112562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:07:03.113997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c43df-55e1-41ac-bbba-60cb3d2fcb27 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:07:03.123793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:07:05.161989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:05.162608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:05.162608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:05.163905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 03:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:07:05.172621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3380f-aa85-4739-88f6-ec0f30b068bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:05.173089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3380f-aa85-4739-88f6-ec0f30b068bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:05.173182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3380f-aa85-4739-88f6-ec0f30b068bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:05.192921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aba3380f-aa85-4739-88f6-ec0f30b068bb None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:07:05.194136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aba3380f-aa85-4739-88f6-ec0f30b068bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:07:05.204168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 03:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:07:05.516937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:07:05.518994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:07:05.533773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad49e54-ba98-4348-9e74-13bcc2e872ab None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:07:05.537728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:07:05.538363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:07:05.538648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:07:05.538922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:07:05.548087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:07:05.549558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b62f6-74e7-4f73-ac38-408ee14052ea service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:07:05.559449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:07:15.187211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=32 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:07:15.554993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=33 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:32.325373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.327104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.327978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.356981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.358906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.369262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:08:32.381478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.382092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.382092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.413686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.415003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.427227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.431991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.447038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3691fb-1c0c-463f-b90f-d136d075a76e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.456325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.461934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.462918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.463410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.464384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:32.464939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.467112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcacde-fbb7-46a3-9f51-748cd70768e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.484397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:32.493581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.508268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c914d61-f816-4b59-b367-bb8c15176cf2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.515837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.516433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.517120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.517396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.517678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.517967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.518365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.518689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.522058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2266ff1-0480-4692-9f7f-b582ef54932b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.556018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.558434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.574411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1543] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.579156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1544] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:32.584442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.588117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcc3df-fbe9-4285-b6be-3d42f0bfd756 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.588699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcc3df-fbe9-4285-b6be-3d42f0bfd756 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.588992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcc3df-fbe9-4285-b6be-3d42f0bfd756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.606879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b2093-893d-44cf-bbe8-3a1e8cf045a6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.626244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5cbcc3df-fbe9-4285-b6be-3d42f0bfd756 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.627502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5cbcc3df-fbe9-4285-b6be-3d42f0bfd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.723019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.724623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.726065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.726632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.727848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.732784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69296c63-f475-472a-9fe2-47f1e9ccecf6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.736300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1545] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.741833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.742548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.743040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.777808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.779081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.780863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1546] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:32.782820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd49bee-1ef0-433c-acf2-bb4dc0491a4e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.783605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd49bee-1ef0-433c-acf2-bb4dc0491a4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.784096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd49bee-1ef0-433c-acf2-bb4dc0491a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.794545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1547] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.794545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.812024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1fd49bee-1ef0-433c-acf2-bb4dc0491a4e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.812452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d8595-372a-481c-9521-490056a7d166 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.814536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1fd49bee-1ef0-433c-acf2-bb4dc0491a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.819178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.820746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.821252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.821740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.822690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:32.822894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1548] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.823457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.824532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.824850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.825127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.825473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:32.825473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:08:32.825473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:08:32.826746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f44d2c-90d8-458a-a5b7-59c9b1f6f2b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.840026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1549] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:32.841846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb0c7-a4a5-4791-93f5-185abf6764d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.842353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb0c7-a4a5-4791-93f5-185abf6764d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.842846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edb0c7-a4a5-4791-93f5-185abf6764d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.845643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.846854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.860450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1550] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.860450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.865499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71a7dd-20f1-4eb6-9015-5386c63031bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.870391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.871070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.871184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2edb0c7-a4a5-4791-93f5-185abf6764d5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.871356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.871623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.872723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.872837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2edb0c7-a4a5-4791-93f5-185abf6764d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.874354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ea9ba-7908-469c-9f17-c73125a1bcfa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.882681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1551] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.889757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.889757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.889757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.890823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1552] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:32.892454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d9ed-3acf-4c3d-96be-80857fca50c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.892741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d9ed-3acf-4c3d-96be-80857fca50c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.893020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d9ed-3acf-4c3d-96be-80857fca50c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.912137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf30d9ed-3acf-4c3d-96be-80857fca50c8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.912549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.913362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf30d9ed-3acf-4c3d-96be-80857fca50c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.913712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.921156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1553] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.922123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1554] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.922633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07e43-e0c3-4303-9a19-023e14e59746 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.922917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07e43-e0c3-4303-9a19-023e14e59746 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.923195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07e43-e0c3-4303-9a19-023e14e59746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.923508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.934272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580a4e6-a5a7-471d-a800-fdb787d440a7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:32.938799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:32.939460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.939741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.940012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.940734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:32.942927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:32.947570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fbf07e43-e0c3-4303-9a19-023e14e59746 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.948826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fbf07e43-e0c3-4303-9a19-023e14e59746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:32.957411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1555] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:32.958705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d1f6-4ace-4189-93e0-2184d6f93f15 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:32.958999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d1f6-4ace-4189-93e0-2184d6f93f15 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:32.959282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d1f6-4ace-4189-93e0-2184d6f93f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:32.974972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1556] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:32.975738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:32.980768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bbd8d1f6-4ace-4189-93e0-2184d6f93f15 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:32.982051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbd8d1f6-4ace-4189-93e0-2184d6f93f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.040000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc63227-dcc9-4962-8a0c-56e4dd0569dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.041718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1557] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.042994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061edd0-c492-4f64-8529-a926177d982b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.043294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061edd0-c492-4f64-8529-a926177d982b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.043568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061edd0-c492-4f64-8529-a926177d982b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.044838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.045784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.046073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.046357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.047098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.048764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7999-9598-4933-892f-9865f9cc01bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.060596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1558] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:33.061896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f733-8b00-4831-ac0d-4aa546d038f0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.062037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4061edd0-c492-4f64-8529-a926177d982b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:33.062224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f733-8b00-4831-ac0d-4aa546d038f0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.062469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f733-8b00-4831-ac0d-4aa546d038f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.063373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4061edd0-c492-4f64-8529-a926177d982b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.072214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.073124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061edd0-c492-4f64-8529-a926177d982b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.083092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061edd0-c492-4f64-8529-a926177d982b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.084531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dd11f733-8b00-4831-ac0d-4aa546d038f0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:33.085750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dd11f733-8b00-4831-ac0d-4aa546d038f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.087927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.088642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.088933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.089212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.089943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.090250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.091852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.094034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1560] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.095085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.095381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.095661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.107680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.110364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1561] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.117974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.118105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:33.118105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.123060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb399da-67dd-4329-b368-0adf9824340c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.124884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1562] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.125512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.128550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.129221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.129507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.129783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.130505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.130811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.132406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.134165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.134904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cf14f-cd8e-4451-b4d8-27c326557f3d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.139557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.140218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.140558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.140833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.141541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.142574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:33.142574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:08:33.142574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:08:33.143818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.158680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1563] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.160123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.170489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa049f-6b4f-4334-813e-19f9130bcee6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.174763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1564] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.174763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.177847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.178518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.178806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.179081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.179779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.180088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.181763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.184736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d770d-d339-4398-991c-1962a9d26cd1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.189282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.190176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.190688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.190830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] PATH_INFO: `/v3/projects/8dd05fc495454e9290c277ed4db003d8/users/937fff3ec09e4f8a9a9a5fcdf2027a2b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.193721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1565] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.194654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.205082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cc7be-b434-461a-aeb1-6b2548922779 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.206712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd05fc495454e9290c277ed4db003d8, user_id=937fff3ec09e4f8a9a9a5fcdf2027a2b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.209603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e88be-c1e2-448c-aff5-c799934bba83 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.210564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.211227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.211513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.211791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.212554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.212867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.214563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.225709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1566] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:33 2022] PUT /identity/v3/projects/8dd05fc495454e9290c277ed4db003d8/users/937fff3ec09e4f8a9a9a5fcdf2027a2b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:33.226700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.226991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.227395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.229673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1567] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.230534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.240580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0b0c2-b2e2-4b13-beea-7c181f8e7410 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.248559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:33.249977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.298712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.299469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.299682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.299972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.300867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.301912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:33.301912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:08:33.301912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:08:33.303116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.315561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1568] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.316496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.325985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007f75d-dadd-45c2-a6fc-9c432e25dda8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.330756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.331360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1569] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.332110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.332110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.332110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.332712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.332950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.333264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.335204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.347064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1570] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.347877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.357253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6380f-61fb-4314-a2a0-f5ae13210a8e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.399432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617fbf7-c2ed-4d5d-a96f-36df19fcfbd9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.404572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.405245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.405534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.405819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.406523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.408483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09e4bc-d6b9-4e1c-bd29-cc4a167fb50e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.412715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.413418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.413701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.413968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.414697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.416331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.426717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1571] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:33.427643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.437439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1572] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.438603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.439054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.439242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d306-ad03-425a-b5d5-ccdff8867200 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.439348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.456297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:33.457587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:33.479949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.491131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.498109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.498835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.499154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.499416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.500149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.502261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.524122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1573] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:33.524122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.531216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da8ff-747a-4269-be60-4beb0e875a37 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.536038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.536561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1574] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.536744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.537035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.537284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.537382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.538006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.539941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.568807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1575] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.569636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.604121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f85a4-bdb1-4efd-9c51-33aed9a45bf5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.630999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5b071-21e9-4148-8a16-0f600118ff24 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.637484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.637706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.637860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.638155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.638982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.641266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.661509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.662216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.662497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.662774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.662983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1576] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.663503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:33.663623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.663895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.665462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.674868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b221ed2-d5e4-4369-a0ee-ffd4b5d52da4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.677347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1577] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:33.678022 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.683260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.684723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.685017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.685459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.686208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.687388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72221e56-00b7-4996-b05c-7948dc935826 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.688206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.692060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.692749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.693034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.693342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.694063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.695649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.706820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1578] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:33.707674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.717885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e341f3-92a5-408d-9080-f609ab8cbd59 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.722700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.723643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.770024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.778334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.779007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.779333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.779612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.779754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713da80-54d5-4754-9758-b40d23ed47c9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.780369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.782587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.783892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.784818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.785119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.785429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.795115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.796619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.805726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1580] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.807117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1581] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:33.810076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.810208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.820230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22d63-213a-43c2-92e2-118fd9587643 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.820500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77968-2920-44af-a4e2-83e42b96d3e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.824907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.825583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.825704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.825858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.826124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.826495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.826784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.826892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.827060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.827762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.828763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.829814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.851924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1582] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.852856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.853753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.854626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.865268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5414c-9063-4ccd-a87d-d900c8a03bd0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.866758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8387-da39-493b-91f2-d711a5c4766d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.869824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.870486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.870769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.871063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.871740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.871966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.873192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.873321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.873559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.874027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.874289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.876003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.879164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:33.889868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1584] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:33.890721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.902665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b737c-6658-45d7-be89-516dd59e8c4e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:33.908748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:33.912627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1585] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:33.912774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:33.912774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:33.912774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] PATH_INFO: `/v3/projects/febf273149c744e08e0f7ea65e44e0d9/users/5bb9100dcf2a4522840d726d800fe013/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:33.913064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.929746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febf273149c744e08e0f7ea65e44e0d9, user_id=5bb9100dcf2a4522840d726d800fe013, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:33.930915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:33.930915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:08:33.930915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:08:33.933421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:33.949838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1586] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:33 2022] PUT /identity/v3/projects/febf273149c744e08e0f7ea65e44e0d9/users/5bb9100dcf2a4522840d726d800fe013/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:33.950794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:33.990027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735326d7-71e3-4fed-843f-e6447b59a2c7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.028016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c620e-c860-4fe5-815f-adb54f3cec35 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.034239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.035302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.035622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.035959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.036780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.038674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.054154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1587] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.055048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.064440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.065133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.065421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.065704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.066023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ae956-f607-4d02-be0e-cc422a40f616 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.067349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.069367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.071346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.072020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.072303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.072616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.073334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.075591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.087705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1588] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.088908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.093248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1589] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:33 2022] POST /identity/v3/users => generated 279 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:34.094158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.111959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6012-1680-49be-b54b-f918babb1891 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.115923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89965b-4f98-478d-9fbe-ebbf0a3cc873 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.118812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.119596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.119843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.120271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.121200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.123032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.127074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.127815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.128100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.128502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.129286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.131650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27eb076-4e58-4fda-9939-56b52e598556 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.136701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1590] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.137618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.148686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.150084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.150644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.150929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.153536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a971f-fdc8-40f0-a098-e23ce8b37334 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.166085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.168170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.168636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.169070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.169720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:34.170217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.170952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:34.173721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.178748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1592] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:34.179559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.190533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0146a-96ef-468e-b86f-a8893c3e44a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.196191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.197044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.197334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.197610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.198350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.199967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.202875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1593] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/users => generated 284 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:34.203867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.212810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.216676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695315c7-ed22-43e3-9cd6-7604373ebf3d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.216831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.222376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.223029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.223764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.223907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.224740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.226662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.228252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53c49-7d6a-414f-a89e-1751a32a0689 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.232946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.233702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.234172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.234517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.235302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.236976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.240448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.241860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.249765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.251955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:34.253781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.257129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ac108-f5f2-4f69-a3d8-3b4fad34164d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.265159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff13a74-a645-4c49-be2d-ae12cec0c8c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.265462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.266419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.267094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.267469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] PATH_INFO: `/v3/projects/8c102f5ee47e426c8f4e0ced15f60bee/users/e3ae9f551626460e864561c613d5f273/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.271970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.272706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.272991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.273263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.273996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.275760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.283993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c102f5ee47e426c8f4e0ced15f60bee, user_id=e3ae9f551626460e864561c613d5f273, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.287040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.287958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.288872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.290659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:34.308178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1598] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:34 2022] PUT /identity/v3/projects/8c102f5ee47e426c8f4e0ced15f60bee/users/e3ae9f551626460e864561c613d5f273/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:34.309450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.358222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079761-0b7e-4758-97c6-eac5ab715465 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.388443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e60d6-fa09-49f7-9228-b2ad69aabc69 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.391345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:34.396151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.397134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.397428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.397708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.398460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.401005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.416606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.418000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.430249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f40b7-ee6a-48dd-8a68-d2ff2709c471 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.434543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.435215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.435497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.435772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.436275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.437289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.437816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.438112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] PATH_INFO: `/v3/projects/7262f17284a44afaa2b5d2d2db0a3ef6/users/8bd2fc9f3216425cbcbbed84b7815529/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.446620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.448454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.456579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7262f17284a44afaa2b5d2d2db0a3ef6, user_id=8bd2fc9f3216425cbcbbed84b7815529, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.459498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.461020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1600] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:34.461918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.472311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b26242-fa3d-40a2-9404-2aa82e6e4e12 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.477623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1601] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:34 2022] PUT /identity/v3/projects/7262f17284a44afaa2b5d2d2db0a3ef6/users/8bd2fc9f3216425cbcbbed84b7815529/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:34.478738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.543822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.544795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.545099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.545393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] PATH_INFO: `/v3/projects/9ddc22cb64464d55ade43a138968b106/users/73c8bb2ba1fa4cb7877ab4d59c5f1294/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.551087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165daa9-6e0f-42a5-8586-057fdd8c771d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.556245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.557080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.557431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.557872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.558682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.560389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.563623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddc22cb64464d55ade43a138968b106, user_id=73c8bb2ba1fa4cb7877ab4d59c5f1294, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.566599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cef9cd-972c-4497-ab78-3bee31e54fe8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.571476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.572516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.584478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefdcd-b616-449e-be06-38ff1087ed51 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.588242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1603] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:34 2022] PUT /identity/v3/projects/9ddc22cb64464d55ade43a138968b106/users/73c8bb2ba1fa4cb7877ab4d59c5f1294/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:34.589672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.590052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.590383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.611086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:34.612390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:34.649707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:34.662324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.663261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.663522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.663793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] PATH_INFO: `/v3/projects/5badbc0000cf41beb6fd192462d4eda8/users/b778c9e1d53c44edb54b0c94c458e20c/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.676503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1604] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:34.677674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.685648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5badbc0000cf41beb6fd192462d4eda8, user_id=b778c9e1d53c44edb54b0c94c458e20c, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.688344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6b5c5-d91f-4849-bb98-8c5631e4bf79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.696317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb25de-ae7d-44f4-be69-0c99a3bd246c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.701199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.701884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.702169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.702440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.703173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.704828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.717706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1605] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:34 2022] PUT /identity/v3/projects/5badbc0000cf41beb6fd192462d4eda8/users/b778c9e1d53c44edb54b0c94c458e20c/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:34.717914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1606] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:34.718040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733f18-d62f-42d9-9fa6-c3ffd5ce50e7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.718040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733f18-d62f-42d9-9fa6-c3ffd5ce50e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.718040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733f18-d62f-42d9-9fa6-c3ffd5ce50e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.718292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.731731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4cc7c-fb87-454d-bab4-c68dd8173064 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.740592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b6733f18-d62f-42d9-9fa6-c3ffd5ce50e7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:34.741825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6733f18-d62f-42d9-9fa6-c3ffd5ce50e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:34.775134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:34.798985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.799818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.800208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.800451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] PATH_INFO: `/v3/projects/90efa0c9d5be417799653fbb9b617b8c/users/bd0d9563f0ac4b9eb7bc2de9dd24a561/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.816947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90efa0c9d5be417799653fbb9b617b8c, user_id=bd0d9563f0ac4b9eb7bc2de9dd24a561, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:34.821925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1607] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:34.822079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:34.822325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e602e-be51-4536-8352-5a53a5b87193 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.822325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e602e-be51-4536-8352-5a53a5b87193 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.822325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e602e-be51-4536-8352-5a53a5b87193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.839822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:34 2022] PUT /identity/v3/projects/90efa0c9d5be417799653fbb9b617b8c/users/bd0d9563f0ac4b9eb7bc2de9dd24a561/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:34.850122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:34.877160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d00e602e-be51-4536-8352-5a53a5b87193 None None] MFA Rules not processed for user `73c8bb2ba1fa4cb7877ab4d59c5f1294`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:34.879096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d00e602e-be51-4536-8352-5a53a5b87193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:34.933588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fefedc-f464-4c5f-9932-a120016da3b7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:34.938510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:34.939357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.939788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.939981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.967938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1609] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:34.969535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:34.969535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:34.969744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:34.995257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:34.996521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:35.081058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.082847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1610] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:35.083770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.084187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.092525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:35.109877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1611] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:35.110014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1429f-a645-4741-a90f-202cd0e69dda None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.113083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.121824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.123549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.123837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.124114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] PATH_INFO: `/v3/projects/1e6725d01aa14272a7f01a52e50af592/users/fb0a8cba80784bc4a6dbf7c1d64256d0/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.125456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad6ade-36eb-4aab-8f42-a889d6d69e13 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.137616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.138985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.139472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.139913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.152312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6725d01aa14272a7f01a52e50af592, user_id=fb0a8cba80784bc4a6dbf7c1d64256d0, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.155705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.156132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.157789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d6c30-aaf7-4056-9f10-74c929b280b7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.176955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1612] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:35.178100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1613] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:35 2022] PUT /identity/v3/projects/1e6725d01aa14272a7f01a52e50af592/users/fb0a8cba80784bc4a6dbf7c1d64256d0/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:35.179045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.179801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6fae-2822-4eea-8dc4-7fc31a1cf6ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.180255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6fae-2822-4eea-8dc4-7fc31a1cf6ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.180589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c6fae-2822-4eea-8dc4-7fc31a1cf6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.221091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-363c6fae-2822-4eea-8dc4-7fc31a1cf6ed None None] MFA Rules not processed for user `bd0d9563f0ac4b9eb7bc2de9dd24a561`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:35.223796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-363c6fae-2822-4eea-8dc4-7fc31a1cf6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:35.265027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383d35-1b4a-4f2a-bde3-b507621bd77a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.275153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.276236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.277053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.277430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.279524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.282009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.300045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:35.332201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1614] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:35.333482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.346875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1615] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:35.348832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.349036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.349429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.417661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] MFA Rules not processed for user `fb0a8cba80784bc4a6dbf7c1d64256d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:35.422447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:35.472988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e52a5-087f-48fc-820c-80b6ed1fbf79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.485146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.486259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.486546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.486814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.582968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1616] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:35.583585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.617238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.618582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.635114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:35.636003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.650297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985e73c-f8db-4d03-ab51-b538136b4efc service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.657621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.658414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.659231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.659749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.660571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.663792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.680494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:35.711304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1618] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/projects => generated 357 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:35.711304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.726073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc313d5-19e0-4192-a423-95287c9716cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.795013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450e329-c5ae-4110-8a46-9457f0ce36be admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.801740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.802898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.803184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.803465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.849328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.850275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.850660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.851036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.851894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.854017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.879201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.880919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.892505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1619] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:35.893460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.900019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1620] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/users => generated 292 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:35.901209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.906306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbdf56-beb6-4eb9-ad50-43c48ffe6d3e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.911751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.912594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.912877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.913149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.914084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.915991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.923147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070d29-e884-42d2-a317-4d4b54da6b7a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.937390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.938272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.938689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.939024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.939827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.945169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1621] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:35 2022] POST /identity/v3/users => generated 285 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:35.945646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.948810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.962487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b26a-3f60-4ae1-b46d-d67587181b23 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:35.970566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:35.971143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1622] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:35.971366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:35.971657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:35.972172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:35.973287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:35.975309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:35.976359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.990348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1623] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:35.991943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:35.994583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:35.998693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923e8a-24e2-4c8d-a7d5-f2c10d53ffcd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.014254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.014824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.015237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.015841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] PATH_INFO: `/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4/users/84c5f9a8cb1c4480a321715aa1707d71/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.021516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f95486-46dc-45f4-896e-892a4bf24803 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.029145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.031977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.032493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.033073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] PATH_INFO: `/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd/users/39b63924fd8440f780ac5e28b42d0b43/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.052353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a1d39a789e42b49313a5e3144bf2f4, user_id=84c5f9a8cb1c4480a321715aa1707d71, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.055317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.059786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=34bb5cddd3214bd5a1ebf7fb7dd5eccd, user_id=39b63924fd8440f780ac5e28b42d0b43, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.065438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.083830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1624] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:35 2022] PUT /identity/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4/users/84c5f9a8cb1c4480a321715aa1707d71/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:36.088806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:36.096180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1625] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:35 2022] PUT /identity/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd/users/39b63924fd8440f780ac5e28b42d0b43/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:36.097835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:36.209533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:36.224988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23975634-4831-4c53-896c-91e9f1dcd2aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.238760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.241522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a3b94-9855-4d94-8d3c-2f489917beaf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.242893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.243408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.244628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.246201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.250003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.257452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.259961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.260253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.261927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.263193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.268563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.278589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:36.286210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:36.304442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1627] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:08:36.308057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:36.318132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310a4f7-1313-405a-a501-47a8e0c3ce44 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.327607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.329733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.331245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e0a79-51ed-45f1-9676-318b3b20f1e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.332106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.332508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] PATH_INFO: `/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4/users/84c5f9a8cb1c4480a321715aa1707d71/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.335986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a1d39a789e42b49313a5e3144bf2f4, user_id=84c5f9a8cb1c4480a321715aa1707d71, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.342312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.343232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.343521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.344021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdce66c-035b-42bf-b4ea-e4ee46fdffd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.345171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] PATH_INFO: `/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd/users/39b63924fd8440f780ac5e28b42d0b43/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.352244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34bb5cddd3214bd5a1ebf7fb7dd5eccd, user_id=39b63924fd8440f780ac5e28b42d0b43, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.357986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9550f-b8d1-485c-9b2a-414b54c98e7f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.381992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1628] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:36 2022] PUT /identity/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4/users/84c5f9a8cb1c4480a321715aa1707d71/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:36.385359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.386375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.386695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.396885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:36 2022] PUT /identity/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd/users/39b63924fd8440f780ac5e28b42d0b43/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:36.408013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.408370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.408616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.454571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] MFA Rules not processed for user `84c5f9a8cb1c4480a321715aa1707d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:36.458687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:36.493002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] MFA Rules not processed for user `39b63924fd8440f780ac5e28b42d0b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:36.494034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:36.701896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1630] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:36.725237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:36.778413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1631] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:36 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:08:36.857327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:36.865685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d2462-ae32-48c4-96e2-e4a299f9f8af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:36.873065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:36.873994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:36.876977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:36.878510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:36.913275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:36.918027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:36.970759 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1632] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:37.047649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:37.047649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01221fde-32e3-4abc-bb3a-55e19d2edec5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:37.047649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:37.047649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:37.051773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:37.051773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:37.064396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:37.067509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:37.088160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1633] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:37 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:37.376794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:37.388042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba091-9e5b-4db2-be4c-f114c9cc2cff service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:37.398382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:37.399062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:37.399398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:37.399739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:37.578682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:37.580170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c15093-22bc-4e85-a790-fedbae21371e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:37.600461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1634] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:37 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:37.930069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:38.064784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba9e90-0336-4adb-9e82-4d9aeaac81f5 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:38.071107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:38.072158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:38.072572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:38.072887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] PATH_INFO: `/v3/users/73c8bb2ba1fa4cb7877ab4d59c5f1294` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:38.074256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c8bb2ba1fa4cb7877ab4d59c5f1294)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:38.076786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:38.213446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c8bb2ba1fa4cb7877ab4d59c5f1294'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:38.215666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c8bb2ba1fa4cb7877ab4d59c5f1294'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:38.221535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:38.242592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c8bb2ba1fa4cb7877ab4d59c5f1294'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:38.243868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c8bb2ba1fa4cb7877ab4d59c5f1294'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:38.258795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c8bb2ba1fa4cb7877ab4d59c5f1294'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:38.295632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1635] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:08:37 2022] DELETE /identity/v3/users/73c8bb2ba1fa4cb7877ab4d59c5f1294 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:38.300639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1405df-56a8-4d9d-ae24-708d4556656d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:38.300938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1405df-56a8-4d9d-ae24-708d4556656d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:38.301226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1405df-56a8-4d9d-ae24-708d4556656d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:38.345863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3f1405df-56a8-4d9d-ae24-708d4556656d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:38.349298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f1405df-56a8-4d9d-ae24-708d4556656d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:38.475214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1636] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:38.518034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:38.520518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:38.667841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed09589-f948-44f0-a832-1e71af8a3bef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:38.674059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:38.674992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:38.675753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:38.676130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:38.699675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:38.703223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978bb5e-c638-48d8-929b-05e54c306ef3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:38.759502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1637] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:40.034024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba031-0ef1-4be6-b587-a520b2d941e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:40.034796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba031-0ef1-4be6-b587-a520b2d941e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:40.035180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba031-0ef1-4be6-b587-a520b2d941e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:40.037325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1638] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jun 21 03:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:40.048533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe0dd-7634-401a-8a31-c0090e81b1c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:40.049147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe0dd-7634-401a-8a31-c0090e81b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:40.049342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afe0dd-7634-401a-8a31-c0090e81b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:40.090270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-99afe0dd-7634-401a-8a31-c0090e81b1c0 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:40.091824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99afe0dd-7634-401a-8a31-c0090e81b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:40.224345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1639] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jun 21 03:08:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:40.230060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:40.230350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:40.230633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:40.232024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1640] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jun 21 03:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:40.239946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:40.240292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:40.240658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:40.285768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:40.287487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:40.304780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1641] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:08:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:40.322602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:40.324485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:40.486344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfd881-f8ca-4fd7-9f4d-03eb9818ff4d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:40.496084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:40.496794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:40.497077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:40.497751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:40.520770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:40.521848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:40.551240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:41.085627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:41.087998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:41.139361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e092bc4-18fc-4714-bb71-5e4855298618 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:41.177235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:41.178605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:41.180290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:41.181359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] PATH_INFO: `/v3/projects/9ddc22cb64464d55ade43a138968b106` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:41.189286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ddc22cb64464d55ade43a138968b106)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:41.247891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:41.247891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 21 03:08:41.247891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: Jun 21 03:08:41.247891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:41.443102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ddc22cb64464d55ade43a138968b106', 'request_id': 'req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.450354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:41.466890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ddc22cb64464d55ade43a138968b106', 'request_id': 'req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.491700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ddc22cb64464d55ade43a138968b106', 'request_id': 'req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.521537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] The token cache is being invalidate because project 9ddc22cb64464d55ade43a138968b106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:08:41.522158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.523574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3cc3fe76-d5aa-4c05-be68-d9a597f4d3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.525965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1643] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:08:41 2022] DELETE /identity/v3/projects/9ddc22cb64464d55ade43a138968b106 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:41.529650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:41.546596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649d0a8-50ba-4a72-9cc2-d0cb0df1754e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:41.646558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:41.714349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:41.718781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:41.719324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:41.719863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] PATH_INFO: `/v3/users/84c5f9a8cb1c4480a321715aa1707d71` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:41.730797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c5f9a8cb1c4480a321715aa1707d71)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:41.735765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 03:08:41.735765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 21 03:08:41.735765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: Jun 21 03:08:41.741822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:41.924386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c5f9a8cb1c4480a321715aa1707d71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.925306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c5f9a8cb1c4480a321715aa1707d71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.926223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c5f9a8cb1c4480a321715aa1707d71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.968827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c5f9a8cb1c4480a321715aa1707d71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:41.986580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d189dcf6-6674-497a-b406-c2c87fc74e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c5f9a8cb1c4480a321715aa1707d71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:42.030118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1644] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:08:41 2022] DELETE /identity/v3/users/84c5f9a8cb1c4480a321715aa1707d71 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:43.058003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfe9b-0a4d-4120-ac3b-4d638201fe48 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:43.058487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfe9b-0a4d-4120-ac3b-4d638201fe48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:43.058559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bfe9b-0a4d-4120-ac3b-4d638201fe48 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:43.060280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1645] 10.222.0.16 () {54 vars in 961 bytes} [Tue Jun 21 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:43.083994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d835-fb53-40ab-9149-b4d963e70967 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:43.085084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d835-fb53-40ab-9149-b4d963e70967 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:43.085691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047d835-fb53-40ab-9149-b4d963e70967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:43.127867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c047d835-fb53-40ab-9149-b4d963e70967 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:43.129465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c047d835-fb53-40ab-9149-b4d963e70967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:43.351967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1646] 10.222.0.16 () {58 vars in 1058 bytes} [Tue Jun 21 03:08:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:43.353196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:43.353569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:43.353850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:43.355284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1647] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jun 21 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:43.364584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:43.365085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:43.365597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:43.409621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:43.411481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:43.444771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1648] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:08:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:43.467098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:43.469304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:43.586290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5551d7-f8b8-4d9e-9387-22cebefb49d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:43.592852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:43.593891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:43.594330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:43.594962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:43.608097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:43.610326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb331ad9-51f0-47f8-8ec0-d88451119cda service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:43.644033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1649] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:44.856976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:44.858938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:45.083513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:45.198422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f887b0c-dd5f-45cd-acfe-3bb8ce01092b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:45.215465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:45.216649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:45.217887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:45.218506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] PATH_INFO: `/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:45.221708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a1d39a789e42b49313a5e3144bf2f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:45.224356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:45.359467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a1d39a789e42b49313a5e3144bf2f4', 'request_id': 'req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:45.375984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a1d39a789e42b49313a5e3144bf2f4', 'request_id': 'req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:45.395859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a1d39a789e42b49313a5e3144bf2f4', 'request_id': 'req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:45.418466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] The token cache is being invalidate because project e9a1d39a789e42b49313a5e3144bf2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:08:45.418775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:45.419545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3b3594-ec6b-4566-9d07-c15f338e9ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:45.421615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1650] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:08:44 2022] DELETE /identity/v3/projects/e9a1d39a789e42b49313a5e3144bf2f4 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:45.431382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:45.432282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:45.432815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:45.476251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:45.477932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:45.771616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1651] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:45.780079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:45.780630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:45.781264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:45.870575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:45.876074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:45.908236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:45.920022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1652] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:45.924251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1c068-3f6f-4f17-a743-dd041408d0de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:45.944658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:45.946149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:45.948775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:45.949185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:45.950081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:45.952076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:45.954023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:45.999627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:46.007806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1653] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:08:46.022830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef8972-d2ae-46f4-97c8-7d2106a18538 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:46.047424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:46.058901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.059418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.059851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.062504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:46.074289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:46.185560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1654] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:45 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:46.195849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:46.229914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36deac9-8745-4e74-ae17-16e44e13b073 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:46.433083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:46.435622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.438624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.438917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.441829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:46.446209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:46.481118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:46.494012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1655] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:46 2022] POST /identity/v3/users => generated 289 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:46.499562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:46.522042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf48afe-197f-4f50-a020-4fbd34447d40 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:46.536946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:46.537582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.537954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.538133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.539362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:46.540560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:46.579681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1656] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:46.580139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:46.599545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156472a9-5be2-4aca-8a73-46bd35959c95 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:46.614703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:46.615314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.615603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.615881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.616659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:46.618288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9e3f2-f3f2-4eae-add6-95024f117602 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:46.667018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:46.678888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc66e80-40bc-4794-b0b1-a15f7b4a5b34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:46.691301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1657] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:46.693383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:46.694326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.694632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.694885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] PATH_INFO: `/v3/projects/d48fcccf923f485da5d068e8628b6a58/users/4998107053fd46a09af2f501a1e4cf5d/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.724151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.724307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.724548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.746861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48fcccf923f485da5d068e8628b6a58, user_id=4998107053fd46a09af2f501a1e4cf5d, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:46.762973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d879f4e-7ca3-4c77-b6b5-e54cdd64a9ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:46.794920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1658] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:46 2022] PUT /identity/v3/projects/d48fcccf923f485da5d068e8628b6a58/users/4998107053fd46a09af2f501a1e4cf5d/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:08:46.805126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] MFA Rules not processed for user `5bb9100dcf2a4522840d726d800fe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:46.806385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:46.807943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:46.808286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:46.808604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:46.887174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] MFA Rules not processed for user `4998107053fd46a09af2f501a1e4cf5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:46.890434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:47.012129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:47.031897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1659] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:47.037665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.044596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1660] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:47.054023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.055489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:47.261630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb597bb-cb0c-416f-abec-bf0c9e005702 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:47.274373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:47.275058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:47.275346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:47.275625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:47.287793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5802d6-3b34-4b54-a225-fc879066f135 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:47.303313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:47.305274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:47.313262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:47.314133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:47.314666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:47.315017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:47.315838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:47.320640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:47.344999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1661] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:47.416480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1662] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:47 2022] POST /identity/v3/projects => generated 375 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:47.423095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.441312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bfac-9687-4624-969d-0f4db5372fd0 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:47.564280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:47.565401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:47.566060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:47.566541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:47.567588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:47.570448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:47.580214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:47.631552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.635544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1663] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:47 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:47.653307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f79b7-2aeb-4a03-bb2c-e9cd523654c9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:47.688431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:47.693263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:47.693635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:47.695413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:47.696469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:47.699981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:47.750614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1664] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:47.757329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.779789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42fb6-b3bc-4f0f-a181-a8124dfbc63f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:47.806933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:47.809677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:47.811158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:47.811576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] PATH_INFO: `/v3/projects/75ae2e269c2f4981a9108cef730b760b/users/2dfa4d5d324b4dae82acc093edcaa901/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:47.876602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ae2e269c2f4981a9108cef730b760b, user_id=2dfa4d5d324b4dae82acc093edcaa901, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:47.886353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:47.934760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1665] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:47 2022] PUT /identity/v3/projects/75ae2e269c2f4981a9108cef730b760b/users/2dfa4d5d324b4dae82acc093edcaa901/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:47.938520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:47.953521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2fbc5-6cbb-4096-a8b8-7bceb1663f28 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:48.108036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:48.108920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:48.110182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:48.110182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:48.111558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:48.115323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a6e4d-5979-47be-b237-66c5bd18200d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:48.138471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1666] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:48.139295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:48.166788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3228acd-187c-463e-9276-28dcc5326f96 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:48.205157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:48.206212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:48.206498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:48.206842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] PATH_INFO: `/v3/projects/75ae2e269c2f4981a9108cef730b760b/users/2dfa4d5d324b4dae82acc093edcaa901/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:48.214469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ae2e269c2f4981a9108cef730b760b, user_id=2dfa4d5d324b4dae82acc093edcaa901, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:48.217364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:48.252464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1667] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:48 2022] PUT /identity/v3/projects/75ae2e269c2f4981a9108cef730b760b/users/2dfa4d5d324b4dae82acc093edcaa901/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:48.264310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab2d1-21f5-42b7-acfb-54b346613807 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:48.264732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab2d1-21f5-42b7-acfb-54b346613807 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:48.264960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab2d1-21f5-42b7-acfb-54b346613807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:48.297071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3eaab2d1-21f5-42b7-acfb-54b346613807 None None] MFA Rules not processed for user `2dfa4d5d324b4dae82acc093edcaa901`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:48.298318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3eaab2d1-21f5-42b7-acfb-54b346613807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:48.360550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:48.437688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1668] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:48.439832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:48.445685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:48.593400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35463795-bdd9-49d1-a36f-f52e24016b38 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:48.603308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:48.604258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:48.604258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:48.604579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:48.755312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:48.756866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:48.785186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1669] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:50.936017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:50.937043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:50.937507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:51.010443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] MFA Rules not processed for user `937fff3ec09e4f8a9a9a5fcdf2027a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:51.015717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:51.204463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1670] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:51.218015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:51.228987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:51.351122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6262faf-1283-4e30-9385-8251900c4be9 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:51.357241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:51.358153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:51.358726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:51.359766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:51.366234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:51.371115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:51.374624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b69a81-319e-408b-9e3d-1dac06efa44e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:51.388490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1671] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:51.787297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:51.789090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:52.083170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950ba93-5d55-49a1-b53c-cce3214f6e36 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:52.109524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:52.110733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:52.110933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:52.111216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] PATH_INFO: `/v3/users/fb0a8cba80784bc4a6dbf7c1d64256d0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:52.118180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0a8cba80784bc4a6dbf7c1d64256d0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:52.120188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:52.299109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0a8cba80784bc4a6dbf7c1d64256d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:52.300658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0a8cba80784bc4a6dbf7c1d64256d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:52.301597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0a8cba80784bc4a6dbf7c1d64256d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:52.330640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0a8cba80784bc4a6dbf7c1d64256d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:52.359577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0a8cba80784bc4a6dbf7c1d64256d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:52.378784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:52.391768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1672] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:08:51 2022] DELETE /identity/v3/users/fb0a8cba80784bc4a6dbf7c1d64256d0 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:52.396656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d80-6de0-45c4-84c0-10df78828818 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:52.397183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d80-6de0-45c4-84c0-10df78828818 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:52.397263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7d80-6de0-45c4-84c0-10df78828818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:52.465093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-446d7d80-6de0-45c4-84c0-10df78828818 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:52.470362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-446d7d80-6de0-45c4-84c0-10df78828818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:52.644827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1673] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:52.693012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:52.695089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:52.701279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:52.701573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:52.701904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:52.764119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] MFA Rules not processed for user `e3ae9f551626460e864561c613d5f273`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:52.765434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:52.834659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ee637-87a7-44e4-b814-907cd8fcf341 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:52.839012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:52.839838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:52.840568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:52.840995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:52.861483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:52.864636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b051502-33ed-4fa1-a470-b71100fd0134 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:52.922964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1674] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:52.936977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1675] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:52.951082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adb25e-42af-4f35-a1ef-e89951a12389 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:52.953376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adb25e-42af-4f35-a1ef-e89951a12389 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:52.955189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adb25e-42af-4f35-a1ef-e89951a12389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:52.979784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-48adb25e-42af-4f35-a1ef-e89951a12389 None None] MFA Rules not processed for user `e3ae9f551626460e864561c613d5f273`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:52.981101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48adb25e-42af-4f35-a1ef-e89951a12389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:53.013502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1676] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:53.025860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:53.028079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:53.150177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032bc5-bdca-4ac3-b211-525c4ecf97ba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:53.174412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:53.175238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:53.175719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:53.176153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:53.190862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:53.196319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a49cbc-fc47-42af-9d51-9d38322dc2c8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:53.224938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1677] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:53.398051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:53.398534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:53.398610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:53.504040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] MFA Rules not processed for user `8bd2fc9f3216425cbcbbed84b7815529`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:53.507059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:53.651727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1678] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:53.663984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8322d-76d7-49c3-9bb5-c0884e5c5c06 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:53.664507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8322d-76d7-49c3-9bb5-c0884e5c5c06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:53.664586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8322d-76d7-49c3-9bb5-c0884e5c5c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:53.707258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-67d8322d-76d7-49c3-9bb5-c0884e5c5c06 None None] MFA Rules not processed for user `8bd2fc9f3216425cbcbbed84b7815529`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:53.710983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-67d8322d-76d7-49c3-9bb5-c0884e5c5c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:53.739964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1679] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:53.751990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:53.754409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:53.786427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea7af-1cfb-4267-9455-258fa32e8487 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:53.804320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:53.808641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:53.809095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:53.809383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:53.829830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:53.832918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16472e05-0d2a-4e6b-94f5-0dbc93d633d4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:53.857343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:54.483940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:54.483940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:54.483940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:54.486015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1681] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jun 21 03:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:54.496638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:54.497137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:54.497562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:54.621089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:54.624352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:54.652047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1682] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:08:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:54.698341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:54.700188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:54.708921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:54.722262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78ee76-7d18-436f-903f-bab8b90b4d38 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:54.734606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:54.735582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:54.736063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:54.736514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:54.753111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:54.754865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcbab9-4270-48fb-be78-d7f3f115d65f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:54.776801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:54.950182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:54.951773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:55.009112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10ad5b-b141-41c4-af08-bda8ba4d6233 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:55.023470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:55.025231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.025946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.027152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] PATH_INFO: `/v3/projects/1e6725d01aa14272a7f01a52e50af592` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.030120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6725d01aa14272a7f01a52e50af592)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:55.039049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:55.168730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:55.174544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6725d01aa14272a7f01a52e50af592', 'request_id': 'req-0884770a-7eb6-4f16-a07e-491541cb2120'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:55.191459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6725d01aa14272a7f01a52e50af592', 'request_id': 'req-0884770a-7eb6-4f16-a07e-491541cb2120'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:55.214971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6725d01aa14272a7f01a52e50af592', 'request_id': 'req-0884770a-7eb6-4f16-a07e-491541cb2120'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:55.244623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] The token cache is being invalidate because project 1e6725d01aa14272a7f01a52e50af592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:08:55.246193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:55.248079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0884770a-7eb6-4f16-a07e-491541cb2120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:08:55.250884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1684] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:08:54 2022] DELETE /identity/v3/projects/1e6725d01aa14272a7f01a52e50af592 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:55.255740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2dab3-604c-48f3-befc-85c6b09d7f63 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.256057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2dab3-604c-48f3-befc-85c6b09d7f63 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.256333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2dab3-604c-48f3-befc-85c6b09d7f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.287919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-09d2dab3-604c-48f3-befc-85c6b09d7f63 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:55.289335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-09d2dab3-604c-48f3-befc-85c6b09d7f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:55.475840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b980-1b13-4e5c-aed5-d1c5ea1def2b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.476141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b980-1b13-4e5c-aed5-d1c5ea1def2b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.476607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6b980-1b13-4e5c-aed5-d1c5ea1def2b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.478469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1685] 10.222.0.16 () {54 vars in 912 bytes} [Tue Jun 21 03:08:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:55.499522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.501893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.502796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.510301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1686] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:55.521019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.521499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.521944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.552983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:55.554751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:55.579452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:55.583033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:55.606349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1687] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:08:55.610226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:55.631643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5bcb4-f429-4e2d-805d-84e5e36b895c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:55.648836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:55.649767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.650211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.651155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.652043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:08:55.652516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:55.654298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1486ca-ce33-4e29-9b92-065c3659e9d7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:55.655711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1688] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:55.676513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1689] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:08:55.681367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:55.702134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8630e-0755-497c-be3a-4a4ecbb0ebe9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:55.708331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:55.709341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.710115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.710574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.711580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:55.715763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:55.754732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1690] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/projects => generated 405 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:55.757025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.757815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.758525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.760225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:55.785205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] MFA Rules not processed for user `5bb9100dcf2a4522840d726d800fe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:55.787744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:55.861185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de20b5b-e4ba-45b9-9828-4751f1dfe024 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:55.872033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:55.876930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.877273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.877455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.880010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:55.883266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:55.924781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1691] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:08:55.927371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:55.945014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfff64-5ec7-4bc6-b2e6-21ec06235ed5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:55.950259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:55.951062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:55.951439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:55.951754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:55.952595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:55.954352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:55.976194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1692] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:55.983048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:55.985174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1693] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:56.013916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb88c7-5c17-483b-bb93-43a86cdb1c66 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:56.053206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:56.054203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.054635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.055068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.056003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:56.057918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e4774-fe5b-4ea1-8c1b-2816f252fd68 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:56.063882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:56.071927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:56.087860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1694] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:08:56.089933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb051f9e-3cb6-4a1e-85b5-e8133e095936 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.090419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb051f9e-3cb6-4a1e-85b5-e8133e095936 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.090842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb051f9e-3cb6-4a1e-85b5-e8133e095936 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.092997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1695] 10.222.0.16 () {54 vars in 960 bytes} [Tue Jun 21 03:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:08:56.093936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb051f9e-3cb6-4a1e-85b5-e8133e095936 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:56.110006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb051f9e-3cb6-4a1e-85b5-e8133e095936 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:56.116143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:56.117375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.117820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.118259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] PATH_INFO: `/v3/projects/74b21c631dbf44ba8dc2f3fb3f703ab3/users/9314b5f05ee448ceb5844ff7f411509b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.151658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b21c631dbf44ba8dc2f3fb3f703ab3, user_id=9314b5f05ee448ceb5844ff7f411509b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:56.155103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f5e4b-85a9-4e9e-a10a-9531622c1e8c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:56.180046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1696] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:08:56 2022] PUT /identity/v3/projects/74b21c631dbf44ba8dc2f3fb3f703ab3/users/9314b5f05ee448ceb5844ff7f411509b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:08:56.181893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6a51-df4f-40f9-b69e-f70d924c27ed None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.182362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6a51-df4f-40f9-b69e-f70d924c27ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.182836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e6a51-df4f-40f9-b69e-f70d924c27ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.205213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f29e6a51-df4f-40f9-b69e-f70d924c27ed None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:56.207583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f29e6a51-df4f-40f9-b69e-f70d924c27ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:56.228961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:56.261478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67df44-c16d-40d4-93f8-fe407709109b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:56.266553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:56.342427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1697] 10.222.0.16 () {58 vars in 1057 bytes} [Tue Jun 21 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:56.344015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369e51-2882-418e-b84a-49cc7bff2f4a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.348346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369e51-2882-418e-b84a-49cc7bff2f4a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.348903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a369e51-2882-418e-b84a-49cc7bff2f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.387979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2a369e51-2882-418e-b84a-49cc7bff2f4a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:56.391128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a369e51-2882-418e-b84a-49cc7bff2f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:56.398464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:56.399341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.399810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.400797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.411445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:56.413734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:56.430262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:56.431638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:56.451529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6663e3-c077-463c-88a5-58c38ca8f03e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:56.467917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:56.470061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.470540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.471014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.475913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:56.493422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:56.497077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:56.515829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1699] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:56.539943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1700] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:56.757446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:56.792889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97211cbc-a4bc-47f2-ac88-ca0ed5d19242 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:56.803334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:56.804016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.804308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.804619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.810141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:56.811297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:56.812252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:56.844792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:56.846913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:56.878097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1701] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:56.888088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] MFA Rules not processed for user `b778c9e1d53c44edb54b0c94c458e20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:56.889746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:57.024453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1702] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:08:57.601887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:57.603849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:57.674279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:08:57.677376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:57.854919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac633d-18ef-42b2-acb7-b8eef293e07c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:57.864338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:57.866461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:57.867696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:57.868686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:57.874590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:57.934620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:57.956470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3339f-1794-4429-a20b-886045e4d3ac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:08:57.992124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:08:58.002650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:58.003226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:58.003647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:58.056192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:58.064455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4eb7f-9d2d-43dc-b86b-e0a042a31f0c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:58.104784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:08:58.106652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a6511c-b3d1-4cce-9cc0-686ab40eadd9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:08:58.119020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1703] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:08:58.149824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1704] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:08:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:08:58.907279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2152-c165-4c94-ab54-d0bb0e205a69 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:08:58.907842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2152-c165-4c94-ab54-d0bb0e205a69 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:08:58.907947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae2152-c165-4c94-ab54-d0bb0e205a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:08:58.959795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5ae2152-c165-4c94-ab54-d0bb0e205a69 None None] MFA Rules not processed for user `8bd2fc9f3216425cbcbbed84b7815529`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:08:58.961876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5ae2152-c165-4c94-ab54-d0bb0e205a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:08:59.015603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:08:59.208514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1705] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:08:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:00.110906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:00.111519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:00.111519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:00.142717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] MFA Rules not processed for user `937fff3ec09e4f8a9a9a5fcdf2027a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:00.144144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:00.324486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1706] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:00.518745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:00.519600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:00.519919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:00.611807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] MFA Rules not processed for user `9314b5f05ee448ceb5844ff7f411509b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:00.616070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:00.770578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1707] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:00.777664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:00.794565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31df610-e4c2-4d6c-8414-cea0fc3d69d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:00.805004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:00.806009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:00.806583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:00.807021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:00.808090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:00.810226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:00.900320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1708] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:00 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:00.905607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:00.928620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f43db9-cae4-4839-b9b8-18c3ce94d447 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:01.039093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:01.039834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:01.040147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:01.040458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:01.041228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:01.043163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:01.074252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1709] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:00 2022] POST /identity/v3/users => generated 309 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:01.078567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:01.096625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6129bd-d714-43d3-9d48-6cc3e754dee6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:01.108635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:01.109935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:01.110813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:01.111543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:01.113814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:01.118119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:01.166200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:01.167963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1710] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:01.185917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f078e8-cbe2-42b4-b61c-1c8927e98086 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:01.202233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:01.219574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:01.219917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:01.220211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] PATH_INFO: `/v3/projects/2ecb15ed53414b118449bc41a291fa8f/users/badbde1479c14181b78420015158efbb/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:01.270695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecb15ed53414b118449bc41a291fa8f, user_id=badbde1479c14181b78420015158efbb, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:01.275504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:01.300457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1711] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:01 2022] PUT /identity/v3/projects/2ecb15ed53414b118449bc41a291fa8f/users/badbde1479c14181b78420015158efbb/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:01.303087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:01.318121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bfc8-d15f-4369-b61c-6e25e39ab46d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:01.504387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:01.507668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:01.508538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:01.509284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:01.510433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:01.515256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f04c1-36c3-405b-8a97-b8a46d312dae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:01.554819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1712] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:01.558089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:01.581818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34808ee5-7bc0-44cd-b6a5-2554e86821ae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:01.601316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:01.602594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:01.603116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:01.603590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] PATH_INFO: `/v3/projects/2ecb15ed53414b118449bc41a291fa8f/users/badbde1479c14181b78420015158efbb/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:01.618438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecb15ed53414b118449bc41a291fa8f, user_id=badbde1479c14181b78420015158efbb, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:01.624258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821be92-469e-40b6-87d2-d488369e997c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:01.653789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1713] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:01 2022] PUT /identity/v3/projects/2ecb15ed53414b118449bc41a291fa8f/users/badbde1479c14181b78420015158efbb/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:02.284131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc59c-8966-4399-93fa-eba766dfd7bc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:02.284728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc59c-8966-4399-93fa-eba766dfd7bc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:02.284872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc59c-8966-4399-93fa-eba766dfd7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:02.342653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-123dc59c-8966-4399-93fa-eba766dfd7bc None None] MFA Rules not processed for user `b778c9e1d53c44edb54b0c94c458e20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:02.352780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-123dc59c-8966-4399-93fa-eba766dfd7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:02.470104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1714] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:03.219719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb87a-f853-4d3a-85b6-ddcd8fa23704 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:03.222258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb87a-f853-4d3a-85b6-ddcd8fa23704 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:03.222696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02beb87a-f853-4d3a-85b6-ddcd8fa23704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:03.248900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:03.260506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-02beb87a-f853-4d3a-85b6-ddcd8fa23704 None None] MFA Rules not processed for user `badbde1479c14181b78420015158efbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:03.263726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02beb87a-f853-4d3a-85b6-ddcd8fa23704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:03.387103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1715] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:04.970157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:04.971290 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:04.971784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:04.980084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1716] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:09:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:09:05.005029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:05.005613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:05.005755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:05.084434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:05.088146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:05.267584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1717] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:09:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:05.284815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:05.287680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:05.316836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:05.318345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:05.320090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:05.321383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92599d-0d8c-473b-b578-e452640e4147 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:05.329947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:05.331005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:05.331645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:05.332431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:05.344351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:05.346029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baeef0-9173-4061-a074-8523f69627a2 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:05.359025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1718] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:05.516490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a2ff-3ad0-4933-94a7-2df21e980a34 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:05.552640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:05.554119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:05.554599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:05.555080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] PATH_INFO: `/v3/users/bd0d9563f0ac4b9eb7bc2de9dd24a561` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:05.557719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0d9563f0ac4b9eb7bc2de9dd24a561)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:05.565620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:05.708830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:05.731425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:05.731718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:05.732036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:05.749679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0d9563f0ac4b9eb7bc2de9dd24a561'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:05.751572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0d9563f0ac4b9eb7bc2de9dd24a561'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:05.754140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0d9563f0ac4b9eb7bc2de9dd24a561'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:05.783425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:05.784515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0d9563f0ac4b9eb7bc2de9dd24a561'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:05.785298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:05.807191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f73ec5dc-575e-4b23-b950-e7f138cbd001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0d9563f0ac4b9eb7bc2de9dd24a561'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:05.853927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1719] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:05 2022] DELETE /identity/v3/users/bd0d9563f0ac4b9eb7bc2de9dd24a561 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:05.855968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:05.855968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:05.856316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:05.897595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:05.902523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:05.948357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1720] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:09:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:06.069073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1721] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:06.097074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:06.098893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:06.231516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:06.234162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:06.237034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de25726-760c-4061-8a07-ffd55c0b2336 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:06.373271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:06.374479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:06.385765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:06.386753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:06.423041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:06.426669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:06.480489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1722] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:06.499958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b20f29-d389-47e0-8d70-736c0aef6ad7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:06.514154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:06.518568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:06.519142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:06.519504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:06.657713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:06.687122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:06.691742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:06.714180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1723] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:09:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:09:07.934473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:07.936047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:07.979252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf79124-14bf-4f11-ac0f-9cd88f33489e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:08.001112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:08.004455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:08.005049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:08.005688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] PATH_INFO: `/v3/projects/90efa0c9d5be417799653fbb9b617b8c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:08.007434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90efa0c9d5be417799653fbb9b617b8c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:08.013315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:08.103446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90efa0c9d5be417799653fbb9b617b8c', 'request_id': 'req-f716a0fa-a0a3-4163-854c-8f0e4251839d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.105081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:08.112094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90efa0c9d5be417799653fbb9b617b8c', 'request_id': 'req-f716a0fa-a0a3-4163-854c-8f0e4251839d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.123835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90efa0c9d5be417799653fbb9b617b8c', 'request_id': 'req-f716a0fa-a0a3-4163-854c-8f0e4251839d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.149042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] The token cache is being invalidate because project 90efa0c9d5be417799653fbb9b617b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:08.149539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.150333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f716a0fa-a0a3-4163-854c-8f0e4251839d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.152653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1724] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:07 2022] DELETE /identity/v3/projects/90efa0c9d5be417799653fbb9b617b8c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:08.156617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:08.172745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d73583-6894-4834-aa7d-da813c2b1d5c service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:08.253006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:08.254485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:08.254485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:08.254899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] PATH_INFO: `/v3/users/39b63924fd8440f780ac5e28b42d0b43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:08.256353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b63924fd8440f780ac5e28b42d0b43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:08.259469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:08.379596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b63924fd8440f780ac5e28b42d0b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.380742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b63924fd8440f780ac5e28b42d0b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.381606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b63924fd8440f780ac5e28b42d0b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.398638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b63924fd8440f780ac5e28b42d0b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.410107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de92a015-1911-405e-8aee-df7518819522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b63924fd8440f780ac5e28b42d0b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:08.432886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1725] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:08 2022] DELETE /identity/v3/users/39b63924fd8440f780ac5e28b42d0b43 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:08.710996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:08.712239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:08.712761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:08.716637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1726] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:09:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:09:08.729823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:08.730244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:08.730625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:08.761914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] MFA Rules not processed for user `beb0197dc69744d69c47ba294b9a4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:08.763190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:08.846898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:08.849277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:08.860851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1727] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:09:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:08.933867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:08.935779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:08.937542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbde85-1348-430f-a8d2-e49e01e18200 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:08.943421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:08.945951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:08.946481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:08.946894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] PATH_INFO: `/v3/users/9314b5f05ee448ceb5844ff7f411509b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:08.948724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9314b5f05ee448ceb5844ff7f411509b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:08.951433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:08.999714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc8c15-0e8f-4bd4-8985-b7248ca0645a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:09.018211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:09.022905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:09.028642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:09.029149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:09.062176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:09.085331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:09.088799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:09.127050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1728] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:09.137754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9314b5f05ee448ceb5844ff7f411509b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:09.156079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9314b5f05ee448ceb5844ff7f411509b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:09.242500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9314b5f05ee448ceb5844ff7f411509b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:09.244610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9314b5f05ee448ceb5844ff7f411509b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:09.266532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9314b5f05ee448ceb5844ff7f411509b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:09.322068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1729] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:08 2022] DELETE /identity/v3/users/9314b5f05ee448ceb5844ff7f411509b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:09.923221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:09.943763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b988f-d089-4c32-afc8-943d51ccfed9 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.048073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.050086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.050773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.051302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] PATH_INFO: `/v3/projects/74b21c631dbf44ba8dc2f3fb3f703ab3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.052878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b21c631dbf44ba8dc2f3fb3f703ab3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.055539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:10.099299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:10.100892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:10.128065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b21c631dbf44ba8dc2f3fb3f703ab3', 'request_id': 'req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.128889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24e8f0-75dd-4117-910f-a9043a34fea3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.135691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b21c631dbf44ba8dc2f3fb3f703ab3', 'request_id': 'req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.155328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b21c631dbf44ba8dc2f3fb3f703ab3', 'request_id': 'req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.178100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] The token cache is being invalidate because project 74b21c631dbf44ba8dc2f3fb3f703ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:10.178491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.179233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.181274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1730] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:09 2022] DELETE /identity/v3/projects/74b21c631dbf44ba8dc2f3fb3f703ab3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:10.186502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:10.188204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:10.235740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:10.246747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.248261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.249528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.249853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] PATH_INFO: `/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.251498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34bb5cddd3214bd5a1ebf7fb7dd5eccd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.254352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:10.291146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fdfd7-c832-43a1-acc0-6160eaabdbed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.297200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.299036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.299476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.299854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] PATH_INFO: `/v3/users/badbde1479c14181b78420015158efbb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.301375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badbde1479c14181b78420015158efbb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.303711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:10.316251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34bb5cddd3214bd5a1ebf7fb7dd5eccd', 'request_id': 'req-195d2144-2a5c-440a-815e-fcc399d46c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.323656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34bb5cddd3214bd5a1ebf7fb7dd5eccd', 'request_id': 'req-195d2144-2a5c-440a-815e-fcc399d46c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.331843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34bb5cddd3214bd5a1ebf7fb7dd5eccd', 'request_id': 'req-195d2144-2a5c-440a-815e-fcc399d46c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.381582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] The token cache is being invalidate because project 34bb5cddd3214bd5a1ebf7fb7dd5eccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:10.381915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.382695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-195d2144-2a5c-440a-815e-fcc399d46c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.385824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1731] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:10 2022] DELETE /identity/v3/projects/34bb5cddd3214bd5a1ebf7fb7dd5eccd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:10.389411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195224cd-215c-4143-95cc-f993b2303bc9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.389789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195224cd-215c-4143-95cc-f993b2303bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.390150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195224cd-215c-4143-95cc-f993b2303bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.431097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-195224cd-215c-4143-95cc-f993b2303bc9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:10.432780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-195224cd-215c-4143-95cc-f993b2303bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:10.521627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1732] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:10.529330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:10.530803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.531199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.532200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.556717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:10.558140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:10.559053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badbde1479c14181b78420015158efbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.559728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badbde1479c14181b78420015158efbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.560343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badbde1479c14181b78420015158efbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.570145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1733] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:10.574225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:10.578985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badbde1479c14181b78420015158efbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.590133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a11a3-9be0-4606-9f62-69757ec61d21 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.606147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.606934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.607225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.607504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.608394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:10.608542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badbde1479c14181b78420015158efbb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:10.608732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.610390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:10.628569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1734] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:10.629579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:10.630684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1735] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:10 2022] DELETE /identity/v3/users/badbde1479c14181b78420015158efbb => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:10.643401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad50ec1-dd34-411b-8196-38870d8eade9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.754258 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.755023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.755313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.755661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.756289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.758280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:10.777839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:10.814891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1736] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:10 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:10.816041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:10.837557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2cbc7-e6a5-4846-8110-829746c899c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:10.982544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:10.983665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:10.984200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:10.984750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:10.986081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:10.990690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.009427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.017358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:11.033640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f969b-c1b3-44a2-adf9-7321fbfb7211 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.042869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1737] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:10 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:11.043081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.043081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.043081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.043081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] PATH_INFO: `/v3/projects/2ecb15ed53414b118449bc41a291fa8f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.043963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecb15ed53414b118449bc41a291fa8f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.044322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.047150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.056444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe36929-7557-48bd-90ee-5719c862e86a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.062295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.063658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.063658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.064006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.064852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.067089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.081970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1738] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:11.086443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.108753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c992d9-c8fa-4739-8a6f-8c08ff5f2bc6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.117966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.118657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.118941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.119213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.119950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.121577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.142079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1739] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:11.145100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.170728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecb15ed53414b118449bc41a291fa8f', 'request_id': 'req-6e287b89-405d-4d71-bd70-6eca5729bae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:11.173631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a8d80-8a9f-4dd4-9368-8e4549910cc9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.182104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecb15ed53414b118449bc41a291fa8f', 'request_id': 'req-6e287b89-405d-4d71-bd70-6eca5729bae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:11.199495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecb15ed53414b118449bc41a291fa8f', 'request_id': 'req-6e287b89-405d-4d71-bd70-6eca5729bae6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:11.227132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] The token cache is being invalidate because project 2ecb15ed53414b118449bc41a291fa8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:11.227691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:11.230710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6e287b89-405d-4d71-bd70-6eca5729bae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:11.237748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1740] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:11 2022] DELETE /identity/v3/projects/2ecb15ed53414b118449bc41a291fa8f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:11.241536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02e341-4a08-4bab-a7d6-bb33e9fadb57 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.242044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02e341-4a08-4bab-a7d6-bb33e9fadb57 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.243222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02e341-4a08-4bab-a7d6-bb33e9fadb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.307278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.308625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.308865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bb02e341-4a08-4bab-a7d6-bb33e9fadb57 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:11.309523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.309769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] PATH_INFO: `/v3/projects/60816105e4fd4b6ea515ee17ce10dce2/users/64dfcfe2552c406fb9dbba27a9a80ad7/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.310115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bb02e341-4a08-4bab-a7d6-bb33e9fadb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:11.330958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1741] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:11.340640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823264b4-36d0-4043-962c-a37e9201701c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.340640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823264b4-36d0-4043-962c-a37e9201701c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.340862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823264b4-36d0-4043-962c-a37e9201701c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.347666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60816105e4fd4b6ea515ee17ce10dce2, user_id=64dfcfe2552c406fb9dbba27a9a80ad7, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.353766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41fdd1-b1d2-4678-ac5e-e082e0943aa3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.360005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:11.396547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-823264b4-36d0-4043-962c-a37e9201701c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:11.396547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-823264b4-36d0-4043-962c-a37e9201701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:11.397128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1742] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:11 2022] PUT /identity/v3/projects/60816105e4fd4b6ea515ee17ce10dce2/users/64dfcfe2552c406fb9dbba27a9a80ad7/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:11.401730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.402117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.403103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.448864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] MFA Rules not processed for user `64dfcfe2552c406fb9dbba27a9a80ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:11.451202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:11.502426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1743] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:11.503944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823264b4-36d0-4043-962c-a37e9201701c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.525138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823264b4-36d0-4043-962c-a37e9201701c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.532381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.533329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.533485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.533898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.534732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:11.535452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.537243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.550480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1744] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:11.555418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.561299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1745] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:11.566346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.566899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:11.569050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c7cf-a94a-4073-b063-9874d8b64f0e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.573736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.574486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.574860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.575205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.576201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.578237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.581981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddfd8-0047-4735-aa27-cc2eb7f81533 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.586921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.587597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.587885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.588164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.588905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.590874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.618586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1746] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:11.619943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.630543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1747] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:11.631597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.673375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:11.716784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f1fd1-eafe-41a0-b95c-12dbc55459cc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.722212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.722959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.723253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.723583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.724356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.726556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.736112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e3e0-b4dc-4fc8-99c6-2c613c6a95cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.756074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.757121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.757723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.758171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.759062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.760436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1748] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:11.762163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.764480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.794220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52de6-a1c0-4672-a240-7b1a301ff022 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.807501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.808209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.808609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.808895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.809748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.811676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.829437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1749] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:11 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:11.832187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1750] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:11.834365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.836263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.868567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52842-0773-486a-86a3-0cd371e84172 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.869143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3528e3-2ef2-44c0-b7a9-8a292dc341c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.875760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.876775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.877077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.877626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.878750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.880313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.884278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.885257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.885547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.885940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] PATH_INFO: `/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44/users/be6c49821add4d85930725e176fc5ecc/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.903682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1751] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:11.904762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.925958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e1ff1-90e6-4c06-a1c9-9d307acda954 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:11.932014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:11.932839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:11.933143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:11.933325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff0896bf94e4545adbd6da77ecf0d44, user_id=be6c49821add4d85930725e176fc5ecc, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.933607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:11.934373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:11.935997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.936104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:11.969108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1753] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:11 2022] PUT /identity/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44/users/be6c49821add4d85930725e176fc5ecc/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:11.970059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1753] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:11.973482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:11.973995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:12.016545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341817e-4d18-4d72-9a21-80f408d30d0c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:12.031362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:12.170063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e96393-d217-4ec4-9bb0-a890ce4f7836 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:12.195117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:12.196266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.200320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:12.201153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.201376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.201681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.202387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:12.204037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:12.204762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.205117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] PATH_INFO: `/v3/projects/246fc325b1014b25b88da77c3fc29b90/users/792509e581cc412bb38e91783a9a6981/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.228692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1754] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:12.233493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:12.245908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95e6b1-25c3-4958-948b-2cae431bf1fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:12.249007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=246fc325b1014b25b88da77c3fc29b90, user_id=792509e581cc412bb38e91783a9a6981, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:12.253008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50fd10-c415-4266-b163-d64faf9ef45d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:12.257771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:12.258823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.259205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.259577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] PATH_INFO: `/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44/users/be6c49821add4d85930725e176fc5ecc/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.265587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff0896bf94e4545adbd6da77ecf0d44, user_id=be6c49821add4d85930725e176fc5ecc, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:12.270282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47997ebf-e136-4029-b007-8d30a0dd07ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:12.292544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1755] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:11 2022] PUT /identity/v3/projects/246fc325b1014b25b88da77c3fc29b90/users/792509e581cc412bb38e91783a9a6981/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:12.301219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcf4ee-a3e0-430b-837c-b6db38f16084 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.301603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcf4ee-a3e0-430b-837c-b6db38f16084 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.302019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcf4ee-a3e0-430b-837c-b6db38f16084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.311749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1756] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:12 2022] PUT /identity/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44/users/be6c49821add4d85930725e176fc5ecc/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:12.331804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.333141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.334288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.371682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3bdcf4ee-a3e0-430b-837c-b6db38f16084 None None] MFA Rules not processed for user `792509e581cc412bb38e91783a9a6981`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:12.373604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3bdcf4ee-a3e0-430b-837c-b6db38f16084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:12.437242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] MFA Rules not processed for user `be6c49821add4d85930725e176fc5ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:12.438617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:12.535515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1757] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:12.536961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.537261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.537549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.577228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:12.578525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:12.642205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1758] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:12.643434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:12.670824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:12.712989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1759] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Jun 21 03:09:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:12.713902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:12.720301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:12.792313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1ee11-2bba-49bd-9ee9-bffd8d557af0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:12.802030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:12.803744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:12.804095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:12.805251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:12.805495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:12.807800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:12.830739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:12.865312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1760] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:12 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:12.870902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:12.891012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f38dc-5b3f-406f-9303-1e644091ad4f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:12.988675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fec8cd-ca1d-43e4-86e3-5e2db15acb06 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.003143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.005489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.005849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.006201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.007489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.013313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.033221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.034978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.036098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.036430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.065992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1761] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:12 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:13.067708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.070736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:13.205338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.207335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.224385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1762] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.225452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.241611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73a117-2184-4de5-a7a1-10e1bb9d1495 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.248808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.249507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.249839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.250120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.252526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e1dd8-927e-433f-b49e-ed671993e6a3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.259195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.260952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.261458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.261783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.262061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.263800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.297878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1763] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.302028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.305399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:13.326984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.328554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.344223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12638f00-b0fc-4c6c-a75b-ad08e896d60c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.357350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1764] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.358453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.361256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.363664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.364351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.364811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.381020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0845-7b0f-4c0f-93cd-b6eec4cccc35 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.392280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.394117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.406151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.406759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.412070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.412380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.420376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1765] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.422871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.424494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.426291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.436394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1766] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.437774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.450300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac72f58-f739-4d7c-af0d-d9bbac37a7e9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.456664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b5581-dc8f-4e83-b235-08252ca7f37c service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.461129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.463095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.466097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.466532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.480386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.481115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.481459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.481783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.484766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.486531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.492220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.503725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.530435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1767] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:13.531837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.535685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1768] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.536664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.548056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:13.557499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf17aa-2099-40c3-b2db-e2ebc0d2e538 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.573992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e026d2-9d9f-449f-9523-049b3785308f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.577396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.578134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.578380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.578660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.589997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.590716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.591005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.591289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.594515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.601396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.644295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1769] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:13.646488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:13.673884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72b04e-c153-4c9d-877d-a5b81c44f0e1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:13.691549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:13.692586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.692914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.693201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] PATH_INFO: `/v3/projects/a23aa38e7bd8475c92e2b37312a42075/users/e8195b28dbdf4a17a635ae385b676869/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.733314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23aa38e7bd8475c92e2b37312a42075, user_id=e8195b28dbdf4a17a635ae385b676869, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.739090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967ed18-a988-432c-82f4-f633aea4c642 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.781632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:13.783146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:13.789453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1770] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:13 2022] PUT /identity/v3/projects/a23aa38e7bd8475c92e2b37312a42075/users/e8195b28dbdf4a17a635ae385b676869/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:13.808775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:13.809415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:13.810785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:13.895827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1771] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:13.906254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] MFA Rules not processed for user `e8195b28dbdf4a17a635ae385b676869`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:13.909995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:13.932893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:14.230805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1772] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:14.232151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:14.292830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a425d-dd73-4391-80ce-8baaf1d1fd6b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:14.444543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:14.445529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:14.445965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:14.446558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:14.447468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:14.449711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:14.506054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1773] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:14 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:14.511277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:14.525200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4d84c-89ea-47f6-90bb-14c2e30cd094 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:14.631076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:14.631785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:14.632072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:14.632351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:14.633095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:14.635077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:14.691357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1774] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:14 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:14.693916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:14.706987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a24034-b905-4da7-b4a0-0cd4ec734f44 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:14.728965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:14.731199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:14.731558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:14.732009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:14.733656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:14.738706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:14.794319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1775] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:14.795419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:14.814958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd02cd-e07c-4e2f-bf9d-a2eb16cdd919 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:14.830533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:14.832241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:14.832241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:14.832241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] PATH_INFO: `/v3/projects/9ebe349222ed459cbf4b755344453a72/users/8fca8ecabcf3413a92de2678093ab693/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:14.873476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebe349222ed459cbf4b755344453a72, user_id=8fca8ecabcf3413a92de2678093ab693, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:14.877696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:14.912945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1776] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:14 2022] PUT /identity/v3/projects/9ebe349222ed459cbf4b755344453a72/users/8fca8ecabcf3413a92de2678093ab693/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:14.918680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:14.930188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a04da-4631-45e8-a22f-828453186ff8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:15.044394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:15.045393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:15.046702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:15.047166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:15.048070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:15.050575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639aea-ff5b-4802-91b5-08c20ae1899e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:15.075546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:15.078758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1777] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:15.100216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8bd65-7fe9-47dc-b241-ff9dc5fc29b5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:15.119114 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:15.120133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:15.122721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:15.123030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] PATH_INFO: `/v3/projects/9ebe349222ed459cbf4b755344453a72/users/8fca8ecabcf3413a92de2678093ab693/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:15.137724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebe349222ed459cbf4b755344453a72, user_id=8fca8ecabcf3413a92de2678093ab693, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:15.138615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:15.180957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1778] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:15 2022] PUT /identity/v3/projects/9ebe349222ed459cbf4b755344453a72/users/8fca8ecabcf3413a92de2678093ab693/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:15.195230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386856c-50d1-4756-aa5f-5b7795b34086 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:15.195820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386856c-50d1-4756-aa5f-5b7795b34086 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:15.196232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386856c-50d1-4756-aa5f-5b7795b34086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:15.240630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:15.250742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2386856c-50d1-4756-aa5f-5b7795b34086 None None] MFA Rules not processed for user `8fca8ecabcf3413a92de2678093ab693`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:15.254528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2386856c-50d1-4756-aa5f-5b7795b34086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:15.475847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1779] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:15.489930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:15.491949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:15.650416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5d787-a5f2-4458-a6e2-d9a3ae5575ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:15.661348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:15.662174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:15.662573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:15.662945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:15.686375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:15.688155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0394e29-11a5-4386-895c-7ae3b4bce8d5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:15.729689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1780] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:16.561010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:16.561432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:16.561690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:16.563905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1781] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:09:16.577322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:16.578466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:16.578746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:16.633691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:16.635034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:16.651620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:16.786460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1782] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:09:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:16.805280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:16.807030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:16.851662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22298f-aaed-4962-80ce-03e0a0a1953d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:16.853692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:16.858412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:16.859040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:16.859326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:16.859597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:16.871649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a13841-0e33-403b-b3df-06110a36d798 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:16.871792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:16.873778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b400-d107-47e0-8a88-2444089b428e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:16.876995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:16.877856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:16.878216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:16.878818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:16.903107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:16.970170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:16.971870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806c3f8-8064-48e7-a336-19e1a0c7c5b1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:16.985002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1784] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:17.957380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16f6f-7191-4883-8f3d-00a83a346977 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:17.957966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16f6f-7191-4883-8f3d-00a83a346977 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:17.957966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16f6f-7191-4883-8f3d-00a83a346977 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:17.959326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1785] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:09:17.976828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:17.977257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:17.977939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:18.006435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:18.007515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:18.018589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1786] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:09:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:18.035626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c16f6f-7191-4883-8f3d-00a83a346977 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:18.053045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c16f6f-7191-4883-8f3d-00a83a346977 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:18.058785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:18.060039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:18.060543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:18.060965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:18.071536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:18.073600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:18.086540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:27.997800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=56 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:28.080621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=40 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:37.270942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:37.283432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21086-4a1f-461a-86ac-5e09ca512588 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:37.287710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:37.288599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:37.288827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:37.289191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:37.357383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:37.358909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec970c-f0b3-4f5a-9f6a-2e7b1d0d3429 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:37.368486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1788] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:38.418369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:38.419894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:38.493123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1bfb4-bb72-4ed2-9f3a-aa2839df7b62 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:38.498101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:38.498992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:38.499275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:38.499802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] PATH_INFO: `/v3/users/4998107053fd46a09af2f501a1e4cf5d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:38.501132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4998107053fd46a09af2f501a1e4cf5d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:38.503291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:38.557072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4998107053fd46a09af2f501a1e4cf5d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:38.557700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4998107053fd46a09af2f501a1e4cf5d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:38.574085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4998107053fd46a09af2f501a1e4cf5d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:38.575157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4998107053fd46a09af2f501a1e4cf5d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:38.582917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4998107053fd46a09af2f501a1e4cf5d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:38.597739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1789] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:38 2022] DELETE /identity/v3/users/4998107053fd46a09af2f501a1e4cf5d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:38.602447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:38.602911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:38.602997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:38.620442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:38.621646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:38.684885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1790] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:38.695497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:38.697108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:38.760192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1668d9-5e8d-4dba-a058-43ffe6173d53 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:38.764435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:38.765226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:38.765583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:38.765917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:38.776283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:38.777783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:38.788516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1791] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:39.242109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:39.247587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:39.267120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601959e3-997e-45dc-afc6-9ad24f8d7d9e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:39.272671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:39.273555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.274002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.274649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] PATH_INFO: `/v3/projects/d48fcccf923f485da5d068e8628b6a58` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:39.276219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48fcccf923f485da5d068e8628b6a58)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:39.278287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:39.329037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48fcccf923f485da5d068e8628b6a58', 'request_id': 'req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.335143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48fcccf923f485da5d068e8628b6a58', 'request_id': 'req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.344629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48fcccf923f485da5d068e8628b6a58', 'request_id': 'req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.357261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] The token cache is being invalidate because project d48fcccf923f485da5d068e8628b6a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:39.357978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.358862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.360716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1792] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:39 2022] DELETE /identity/v3/projects/d48fcccf923f485da5d068e8628b6a58 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:39.365214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:39.375597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10d0cc-0507-48a9-89b4-fd82c3a107ea service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:39.395114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:39.440396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:39.441398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.441681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.441956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] PATH_INFO: `/v3/users/2dfa4d5d324b4dae82acc093edcaa901` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:39.443156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dfa4d5d324b4dae82acc093edcaa901)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:39.445184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:39.502899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dfa4d5d324b4dae82acc093edcaa901'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.503629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dfa4d5d324b4dae82acc093edcaa901'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.519205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dfa4d5d324b4dae82acc093edcaa901'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.519807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dfa4d5d324b4dae82acc093edcaa901'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.528603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dfa4d5d324b4dae82acc093edcaa901'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.552456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1793] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:39 2022] DELETE /identity/v3/users/2dfa4d5d324b4dae82acc093edcaa901 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:39.676828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:39.678907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:39.693687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fd060-a4e9-4958-b33d-ea54cbb015e5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:39.717862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:39.738175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:39.740913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:39.765043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:39.766072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.766423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.766866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] PATH_INFO: `/v3/users/792509e581cc412bb38e91783a9a6981` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:39.768157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=792509e581cc412bb38e91783a9a6981)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:39.770800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:39.827280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792509e581cc412bb38e91783a9a6981'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.827929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792509e581cc412bb38e91783a9a6981'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.828565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792509e581cc412bb38e91783a9a6981'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.838265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d27c3-f306-4ce7-b33d-2cb4fbe9c31a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:39.841530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792509e581cc412bb38e91783a9a6981'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.843969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:39.845346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.845638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.845918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] PATH_INFO: `/v3/users/64dfcfe2552c406fb9dbba27a9a80ad7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:39.847359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64dfcfe2552c406fb9dbba27a9a80ad7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:39.850245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:39.851327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cc82dd-e7f9-426d-b01b-73daab423abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792509e581cc412bb38e91783a9a6981'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.868853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1794] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:39 2022] DELETE /identity/v3/users/792509e581cc412bb38e91783a9a6981 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:39.871642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a888a-ddbd-4e3f-8fef-3211d795658d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.871979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a888a-ddbd-4e3f-8fef-3211d795658d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.872709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a888a-ddbd-4e3f-8fef-3211d795658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:39.898211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-290a888a-ddbd-4e3f-8fef-3211d795658d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:39.900192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-290a888a-ddbd-4e3f-8fef-3211d795658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:39.901584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:39.905045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64dfcfe2552c406fb9dbba27a9a80ad7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.905797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64dfcfe2552c406fb9dbba27a9a80ad7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.927508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64dfcfe2552c406fb9dbba27a9a80ad7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.928264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64dfcfe2552c406fb9dbba27a9a80ad7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.936910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64dfcfe2552c406fb9dbba27a9a80ad7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:39.956713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1795] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:39 2022] DELETE /identity/v3/users/64dfcfe2552c406fb9dbba27a9a80ad7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:39.956713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:39.956713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:39.993087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1796] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:39.995629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:39.995629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:39.995823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.019212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:40.020782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.051760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eddece-e45c-4e5c-8fc9-53344e4fd8f8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.059994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.060738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.061231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.061682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] PATH_INFO: `/v3/projects/75ae2e269c2f4981a9108cef730b760b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.063513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ae2e269c2f4981a9108cef730b760b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.065029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:40.066181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.097850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1797] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:40.098371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.100144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.139458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ae2e269c2f4981a9108cef730b760b', 'request_id': 'req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:40.153385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ae2e269c2f4981a9108cef730b760b', 'request_id': 'req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:40.162277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ae2e269c2f4981a9108cef730b760b', 'request_id': 'req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:40.176623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8b028-b4b2-43cb-86db-95eeb9125da5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.182177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] The token cache is being invalidate because project 75ae2e269c2f4981a9108cef730b760b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:40.182481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:40.183134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:40.188495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1798] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:39 2022] DELETE /identity/v3/projects/75ae2e269c2f4981a9108cef730b760b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:40.189363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.190887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.243626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.244937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.244937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.248553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.250669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.285698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8f7aa-1788-44f5-88c7-0f3ba7ed9200 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.299678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.300390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.300707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.300988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.303219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.309774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:40.351003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.360074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811347d-a7eb-4205-8503-70eee0c7e865 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.376064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1799] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:40.377625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed354c3-cad6-45b5-abba-884f0bac3110 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.377871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed354c3-cad6-45b5-abba-884f0bac3110 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.378159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed354c3-cad6-45b5-abba-884f0bac3110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.400705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fed354c3-cad6-45b5-abba-884f0bac3110 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:40.402434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fed354c3-cad6-45b5-abba-884f0bac3110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.418653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1800] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:40.421068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.421574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.422049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.426048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.427532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.452196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:40.453731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.454264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1801] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:40.470843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.481310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1802] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:40.482579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee33be8-08f7-4a7f-b123-6dabb82d2dab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.488722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.489741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.490069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.490304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.491053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:40.491386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.493536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.512979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.521001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1803] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:40.525898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0411f5c-b026-4967-be79-c24ac7304cec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.534082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.534082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.534082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.534751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.535145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.537568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.578003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1804] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:40 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:40.579433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.591797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849967f-0a59-4f5c-924d-9d189bb5ede3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.667964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.668882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.669170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.670712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.670712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.672250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.705648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1805] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:40 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:40.705956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.719011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41a94-f772-4b3f-87e1-6a9ce9c01120 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.724466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.725212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.725508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.725784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.726745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.728255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.740357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1806] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:40.744144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.757111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6ac7d-5627-4f80-ba16-1186176004f9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.761891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.762828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.763190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.764444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.764444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.765935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.777317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1807] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:40.781483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.786875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:40.793019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27b5eb-879a-4ac5-924a-dffca2758619 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.802079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.803017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.803454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.803746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] PATH_INFO: `/v3/projects/2527a2eedd2d4c0bb0ac206456cac688/users/089e8600e4364badbfab72ebcd269e36/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.822251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.823822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2527a2eedd2d4c0bb0ac206456cac688, user_id=089e8600e4364badbfab72ebcd269e36, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.824646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.827564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf594701-07f7-4883-85b3-a3cc6ccbb161 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.846433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1808] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:40 2022] PUT /identity/v3/projects/2527a2eedd2d4c0bb0ac206456cac688/users/089e8600e4364badbfab72ebcd269e36/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:40.850482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8500f7e-1716-4d74-8c4b-16c5b70cc049 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:40.858665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.859015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.859307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.896569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] MFA Rules not processed for user `089e8600e4364badbfab72ebcd269e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:40.897793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.934613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:40.935581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:40.935874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:40.936156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] PATH_INFO: `/v3/projects/246fc325b1014b25b88da77c3fc29b90` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:40.937667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246fc325b1014b25b88da77c3fc29b90)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:40.942664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:40.954443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:40.980002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1809] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:40.980002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:40.980002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:40.993401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03594a9-8c36-4b41-bf50-9be878cad2bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.007176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246fc325b1014b25b88da77c3fc29b90', 'request_id': 'req-a6457426-f534-4d61-8c61-1b650767fa66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.012698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246fc325b1014b25b88da77c3fc29b90', 'request_id': 'req-a6457426-f534-4d61-8c61-1b650767fa66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.019473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246fc325b1014b25b88da77c3fc29b90', 'request_id': 'req-a6457426-f534-4d61-8c61-1b650767fa66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.033155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] The token cache is being invalidate because project 246fc325b1014b25b88da77c3fc29b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:41.033443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.034084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.035720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1810] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:40 2022] DELETE /identity/v3/projects/246fc325b1014b25b88da77c3fc29b90 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:41.036508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.037847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.053908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.054781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.055069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.055348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] PATH_INFO: `/v3/projects/60816105e4fd4b6ea515ee17ce10dce2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.056551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60816105e4fd4b6ea515ee17ce10dce2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.058482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.101692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6457426-f534-4d61-8c61-1b650767fa66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.112822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60816105e4fd4b6ea515ee17ce10dce2', 'request_id': 'req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.119707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60816105e4fd4b6ea515ee17ce10dce2', 'request_id': 'req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.127794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60816105e4fd4b6ea515ee17ce10dce2', 'request_id': 'req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.136054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:41.144270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] The token cache is being invalidate because project 60816105e4fd4b6ea515ee17ce10dce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:41.151627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.153473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.156693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1811] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:40 2022] DELETE /identity/v3/projects/60816105e4fd4b6ea515ee17ce10dce2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:41.159947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.160688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.180801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.180801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.180801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.180801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.181388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.183526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.223411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1812] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:41 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:41.224078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.225797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.268703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4c75-fd4a-4f8a-b336-e83e3acf2a0d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.292005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:41.341662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97d041-6e98-46d2-a66b-ef88d4a5a77d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.347907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.349493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.350071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.350071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] PATH_INFO: `/v3/users/be6c49821add4d85930725e176fc5ecc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.351335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6c49821add4d85930725e176fc5ecc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.353461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.391171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.392141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.392510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.392822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] PATH_INFO: `/v3/users/e8195b28dbdf4a17a635ae385b676869` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.395010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8195b28dbdf4a17a635ae385b676869)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.397078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.424600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6c49821add4d85930725e176fc5ecc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.425398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6c49821add4d85930725e176fc5ecc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.443106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6c49821add4d85930725e176fc5ecc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.443106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6c49821add4d85930725e176fc5ecc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.446990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6c49821add4d85930725e176fc5ecc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.448688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8195b28dbdf4a17a635ae385b676869'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.449355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8195b28dbdf4a17a635ae385b676869'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.449974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8195b28dbdf4a17a635ae385b676869'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.461394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1813] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:41 2022] DELETE /identity/v3/users/be6c49821add4d85930725e176fc5ecc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:41.462277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.463671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.465714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8195b28dbdf4a17a635ae385b676869'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.475767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8195b28dbdf4a17a635ae385b676869'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:41.490015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1814] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:41 2022] DELETE /identity/v3/users/e8195b28dbdf4a17a635ae385b676869 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:41.584117 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724929b3-7434-444b-9bff-8d5992bf5653 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.673525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:41.681276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.682664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.682991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.684194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.685546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.689838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.724113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1815] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:41 2022] POST /identity/v3/users => generated 303 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:41.725339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.727679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.754931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8e19d-b38a-46f6-b335-2cfc7905fd82 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.759783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.760514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.761292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.761859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.762703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.764534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.774964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1816] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:41.779259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.790467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c72a8a-5bc0-4efc-b317-6cf3ea4a1d8f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.795571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.796539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.797199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.797199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] PATH_INFO: `/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2/users/4f29601df0a14b05ac790adc56c26501/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.813455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd8ff34c4f8457b9c1fcfbfe060c4f2, user_id=4f29601df0a14b05ac790adc56c26501, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.816245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.830361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1817] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:41 2022] PUT /identity/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2/users/4f29601df0a14b05ac790adc56c26501/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:41.835047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.846188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1456a0-3591-4f6a-9b6a-44d1fa5facaf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.856306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:41.908628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.909707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.909707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.909999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.910742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.912343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.924884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1818] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:41.931384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.949792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade76e8b-26f5-4e02-a4d7-61cc4dbfde8e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.955045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.958365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.958365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.958365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] PATH_INFO: `/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2/users/4f29601df0a14b05ac790adc56c26501/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.960132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:41.962250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:41.967555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd8ff34c4f8457b9c1fcfbfe060c4f2, user_id=4f29601df0a14b05ac790adc56c26501, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.970280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:41.976476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6ee3b-3544-4c36-a633-1a2376f55c37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:41.982594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:41.983874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:41.984498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:41.984928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] PATH_INFO: `/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:41.986229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff0896bf94e4545adbd6da77ecf0d44)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:41.988198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.008658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1819] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:41 2022] PUT /identity/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2/users/4f29601df0a14b05ac790adc56c26501/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:42.011254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.016675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.049619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff0896bf94e4545adbd6da77ecf0d44', 'request_id': 'req-c12ac43a-1f32-488b-9991-c7852f7679eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.060722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff0896bf94e4545adbd6da77ecf0d44', 'request_id': 'req-c12ac43a-1f32-488b-9991-c7852f7679eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.071291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff0896bf94e4545adbd6da77ecf0d44', 'request_id': 'req-c12ac43a-1f32-488b-9991-c7852f7679eb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.087708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] The token cache is being invalidate because project 9ff0896bf94e4545adbd6da77ecf0d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:42.088327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.089576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c12ac43a-1f32-488b-9991-c7852f7679eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.091598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1820] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:41 2022] DELETE /identity/v3/projects/9ff0896bf94e4545adbd6da77ecf0d44 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:42.094741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac206bf-63ac-4f91-a49e-3aad09eb11f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.095275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac206bf-63ac-4f91-a49e-3aad09eb11f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.095979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac206bf-63ac-4f91-a49e-3aad09eb11f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.108829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1faa-cd07-4b15-8e98-8fa1a3eb6a90 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.126966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fac206bf-63ac-4f91-a49e-3aad09eb11f7 None None] MFA Rules not processed for user `4f29601df0a14b05ac790adc56c26501`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:42.128217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fac206bf-63ac-4f91-a49e-3aad09eb11f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.173032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.173919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.174218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.174479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] PATH_INFO: `/v3/projects/a23aa38e7bd8475c92e2b37312a42075` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.175729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23aa38e7bd8475c92e2b37312a42075)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.177693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.197725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1821] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:42.199011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.199302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.199578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.218434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:42.219655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.231103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23aa38e7bd8475c92e2b37312a42075', 'request_id': 'req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.231849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:42.236509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:42.238773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23aa38e7bd8475c92e2b37312a42075', 'request_id': 'req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.249349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23aa38e7bd8475c92e2b37312a42075', 'request_id': 'req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.264925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] The token cache is being invalidate because project a23aa38e7bd8475c92e2b37312a42075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:42.265456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.265894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.267830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1822] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:42 2022] DELETE /identity/v3/projects/a23aa38e7bd8475c92e2b37312a42075 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:42.268431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.270361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.300231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1823] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:42.301967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.307937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.321418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533c0b5-2eda-4295-8af4-607cd333731d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.326866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.327803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.328081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.328744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] PATH_INFO: `/v3/users/8fca8ecabcf3413a92de2678093ab693` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.329633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fca8ecabcf3413a92de2678093ab693)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.331869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.341918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c27d15-3f44-4d1e-b3fe-fa9eae4f7c38 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.345736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.346480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.346836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.347263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.349278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.392290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fca8ecabcf3413a92de2678093ab693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.393066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fca8ecabcf3413a92de2678093ab693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.394877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fca8ecabcf3413a92de2678093ab693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.416396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fca8ecabcf3413a92de2678093ab693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.426683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b229ad13-bf3f-4fcf-8dd2-831547d5a8a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fca8ecabcf3413a92de2678093ab693'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:42.430572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.432170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.448000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1824] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:42 2022] DELETE /identity/v3/users/8fca8ecabcf3413a92de2678093ab693 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:42.448000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.448000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.448000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.448426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1825] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:42.481738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:42.483177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.529780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.531350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.552028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3dadd-616a-4666-9ee3-557aaafb9bd4 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.600714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1826] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:42.602692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.613846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7203ac-7880-4a69-bde6-5238c5a24de9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.619198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.619894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.620174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.620497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.621308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:42.621679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.623866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.638758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1827] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:42.644245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.660629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fd9e-c477-4459-94c2-8397c75cd1e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.673930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.674609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.674924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.675090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.675298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.676517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.676971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.677369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.678963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.682982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8f0d4-32b4-474b-9ef7-ea0afa5966be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.683695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.713007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:42.740787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1828] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:42.744074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2719a0-ff74-44ba-9901-1404b94f36ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.744378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2719a0-ff74-44ba-9901-1404b94f36ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.744956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2719a0-ff74-44ba-9901-1404b94f36ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.746987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1829] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:09:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:09:42.748845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2719a0-ff74-44ba-9901-1404b94f36ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.768800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:42.796889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.801160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d461195-b9b2-4a3a-8f45-f345aa82f052 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.826878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1830] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:42.828428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.828748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.829032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.854441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:42.863653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:42.900171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2719a0-ff74-44ba-9901-1404b94f36ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.911840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.912382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.912859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.913156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.922788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.922788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.954290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1831] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/users => generated 282 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:42.958441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:42.975524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e7328-3dff-49f2-9bf7-97e5ff0ac0ac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:42.987073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:42.987855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:42.988166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:42.988559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:42.990871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:42.996580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:42.996852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1832] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:09:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:43.010340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1833] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:43.014241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.025930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5fd70-a3e0-4755-9853-5aa3f0862eea None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.035914 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.037099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.037884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.038901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.043502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.044587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.061390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1834] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:43.065144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.078427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69730fd-6d5c-4204-b48e-d45e56a0fb8c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.084972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.086568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.087021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.087641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] PATH_INFO: `/v3/projects/79d68780ab0540febd8d9a35cc8db7b6/users/fb91a6abfb5746bb9b19f67f8b01fe3a/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.107370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.110417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.114019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d68780ab0540febd8d9a35cc8db7b6, user_id=fb91a6abfb5746bb9b19f67f8b01fe3a, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.117129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6425ee5-127c-4385-9e67-3f386c98a5df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.137330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1835] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:43 2022] PUT /identity/v3/projects/79d68780ab0540febd8d9a35cc8db7b6/users/fb91a6abfb5746bb9b19f67f8b01fe3a/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:43.149099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.149516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.149857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.157550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb305-9dd7-4f7d-ab74-f45a43de8005 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.189404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] MFA Rules not processed for user `fb91a6abfb5746bb9b19f67f8b01fe3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:43.190646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.258644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.259467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.259886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.260325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.292589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.298551 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.320005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1836] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:43.320005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.320005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.353824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1837] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:43.358554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.383132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:43.392170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b59f1d-87ad-4758-8d68-333740c36c16 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.397806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.399042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.399670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.400148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] PATH_INFO: `/v3/projects/9ebe349222ed459cbf4b755344453a72` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.402082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebe349222ed459cbf4b755344453a72)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.404478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.472082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebe349222ed459cbf4b755344453a72', 'request_id': 'req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:43.477572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:43.479402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebe349222ed459cbf4b755344453a72', 'request_id': 'req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:43.488530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebe349222ed459cbf4b755344453a72', 'request_id': 'req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:43.507388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] The token cache is being invalidate because project 9ebe349222ed459cbf4b755344453a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:43.507698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:43.508514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d049af5-5f65-4876-a2bc-b8378e7a6b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:43.515068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1838] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:43 2022] DELETE /identity/v3/projects/9ebe349222ed459cbf4b755344453a72 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:43.516844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b965d2-0034-4ebd-943a-8fc0a3b2a0cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.517145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b965d2-0034-4ebd-943a-8fc0a3b2a0cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.517426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b965d2-0034-4ebd-943a-8fc0a3b2a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.528989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8eb7ea-212f-4e99-a0aa-348f00351921 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.542293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-26b965d2-0034-4ebd-943a-8fc0a3b2a0cf None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:43.543470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-26b965d2-0034-4ebd-943a-8fc0a3b2a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.595155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.595836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.596127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.596432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.597380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.599493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.616397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1839] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:43.625689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.625981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.626263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.627020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1840] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/projects => generated 363 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:43.630990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.632674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.644076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:43.645295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:43.698101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7e390-b616-4910-b321-ad52a5525220 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.702744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.703423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.703707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.703980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.704306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1841] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:43.704850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.706881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.708824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.718195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068139b-70f3-41bd-b3a8-8570042410cd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.723305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.723971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.724273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.724568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.725300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:43.725611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.727231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.730479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1842] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:43.732860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.739149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1843] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:09:43.743128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee22f4-f03d-4f08-8733-b06ea3ec7acc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.743596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.748302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.748988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.749273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.749553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.750282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.751910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.753987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f408e-616f-4881-97e2-de8d96c2133b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.758966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.759623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.759913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.760188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.760939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.765860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1844] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:43.765978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.766796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.775303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:43.776717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb4f61-c0a0-4550-a5fa-79899834d82e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.781697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.782579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.782862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.783294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] PATH_INFO: `/v3/projects/cf4b57c59b954c97915e54d3a7b80a68/users/e66c636ab37e43f5aa3d887b98fbc331/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.793874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1845] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:43.798037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.798788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4b57c59b954c97915e54d3a7b80a68, user_id=e66c636ab37e43f5aa3d887b98fbc331, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.801838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.817058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1846] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:43 2022] PUT /identity/v3/projects/cf4b57c59b954c97915e54d3a7b80a68/users/e66c636ab37e43f5aa3d887b98fbc331/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:43.820831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.854407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:43.867017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d3bac-cbb2-47fb-85f2-7b792568d480 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.889353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dd3ae-27bd-4504-ad00-8d2fe5a8f8fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.894115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.894940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.895231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.895510 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.896281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.897936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.908459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1847] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:43.912245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.923171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41b1a3-df31-465d-84ae-9a531085dab2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:43.929366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.930506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.930798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.931082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] PATH_INFO: `/v3/projects/cf4b57c59b954c97915e54d3a7b80a68/users/e66c636ab37e43f5aa3d887b98fbc331/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.933594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4b57c59b954c97915e54d3a7b80a68, user_id=e66c636ab37e43f5aa3d887b98fbc331, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.936754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302ca19-6509-4f44-a43f-73f656594a4d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.938727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:43.939905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.940728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.941401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.942557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:43.945611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:43.957648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1848] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:43 2022] PUT /identity/v3/projects/cf4b57c59b954c97915e54d3a7b80a68/users/e66c636ab37e43f5aa3d887b98fbc331/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:43.959260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:43.959553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:43.959840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:43.975296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1849] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:43.980229 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:43.994661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04331044-fc15-4950-8cdb-0fdc558846f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.009579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] MFA Rules not processed for user `e66c636ab37e43f5aa3d887b98fbc331`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:44.011180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:44.074469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.076330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.076838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.077312 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.078097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.079749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.102109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1850] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:44.123168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.125777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1851] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:44.137515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.139054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:44.156203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1c812-b640-4bb8-9225-ebc5a5f15feb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.184511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.185859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.186346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.186794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.187702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.189724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.202278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1852] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:44.206309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.228651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445249-3e34-41b3-b79e-e1d1a62585ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.237143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.239492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.240039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.240893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] PATH_INFO: `/v3/projects/b532dada1c104aeeb996130e013431b1/users/f6cad538f7c04e4987a87242b81bc0ae/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.254556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b20b3-d557-4a3e-a71f-341c18a05333 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.259400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.260294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.260797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.261123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.265163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:44.273879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b532dada1c104aeeb996130e013431b1, user_id=f6cad538f7c04e4987a87242b81bc0ae, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.274598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.276317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ffdb3d-f08a-4475-83b8-664106e6bc94 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.276594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.291469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:44.308621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1854] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:44 2022] PUT /identity/v3/projects/b532dada1c104aeeb996130e013431b1/users/f6cad538f7c04e4987a87242b81bc0ae/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:44.323642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.323642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.323982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.370423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] MFA Rules not processed for user `f6cad538f7c04e4987a87242b81bc0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:44.372907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:44.415233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.416364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:44.480617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1855] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:44.481946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.533411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5855-bfaa-4372-b629-a8800cda93b4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.541647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.542329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.542934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.542934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.549068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:44.612816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6718d3a-d450-45fc-87e9-82643059b189 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.631682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.633668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.633795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.634354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.634963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.641885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.682014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.686195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.708811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1856] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:44.718189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1857] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:44 2022] POST /identity/v3/projects => generated 381 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:44.723206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.746385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402048c7-3e23-48a6-aae8-a88fe497681b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.854518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.855526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.856040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.856497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.857396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.859938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.904929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1858] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:44 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:44.909097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:44.925424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a48b2-0802-4b44-aae7-6b30ac63f3e1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:44.938728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:44.939868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:44.940755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:44.942480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:44.949499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:44.951225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:44.972328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1859] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:44.976470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:45.001442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526153d4-9b6c-4c44-859e-9a81f50fa0cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:45.009240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:45.009240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.009240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.009240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] PATH_INFO: `/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30/users/cf5eedfd6a3e4e78a65391c399a9db19/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.026335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7089c93f7940c1930f31a1f2b0dc30, user_id=cf5eedfd6a3e4e78a65391c399a9db19, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:45.029131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:45.042825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1860] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:44 2022] PUT /identity/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30/users/cf5eedfd6a3e4e78a65391c399a9db19/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:45.046880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:45.057228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827445-9ece-4a2b-aa42-9b4c7e444fa2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:45.125150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:45.125885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.126232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.126424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.127198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:45.128892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2a37-af5b-42c3-b16a-77ad34307132 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:45.140197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1861] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:45.144963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:45.158082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11c07-7aa4-4a06-a449-79cabfbd73af admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:45.163546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:45.164508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.164795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.165071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] PATH_INFO: `/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30/users/cf5eedfd6a3e4e78a65391c399a9db19/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.167175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7089c93f7940c1930f31a1f2b0dc30, user_id=cf5eedfd6a3e4e78a65391c399a9db19, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:45.170112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:45.173850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:45.184324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1862] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:45 2022] PUT /identity/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30/users/cf5eedfd6a3e4e78a65391c399a9db19/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:45.194433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.194723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.195001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.225602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] MFA Rules not processed for user `cf5eedfd6a3e4e78a65391c399a9db19`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:45.226829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:45.261990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:45.320377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1863] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:45.331998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:45.407849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fb24c-ff6b-4c17-a93c-a8c850f7763a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:45.411799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:45.412641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.412927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.413206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.423006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:45.424584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:45.435505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1864] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:45.542713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:45.553715 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8103f1-a65d-4fd7-a462-665e69de3ba2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:45.558745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:45.559405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:45.559698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:45.560025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:45.650336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:45.651774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b08511-03c2-47e9-8731-4fa746bab668 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:45.660730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1865] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:46.925180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:46.926875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:46.999478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62302a-b796-4adb-8845-89433a487e73 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:47.014292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:47.016146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.016668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.017448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] PATH_INFO: `/v3/users/089e8600e4364badbfab72ebcd269e36` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.019590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089e8600e4364badbfab72ebcd269e36)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:47.023031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:47.119299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089e8600e4364badbfab72ebcd269e36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.120239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089e8600e4364badbfab72ebcd269e36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.145614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089e8600e4364badbfab72ebcd269e36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.154806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089e8600e4364badbfab72ebcd269e36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.174293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089e8600e4364badbfab72ebcd269e36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.200593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1866] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:46 2022] DELETE /identity/v3/users/089e8600e4364badbfab72ebcd269e36 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:47.204629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.205129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.205280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.234770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:47.236178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.377177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1867] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:47.388589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:47.390701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.414099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:47.458607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd34ce-7022-47ff-a7af-5263f1c6f8a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:47.463317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:47.465328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.465518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.465799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.476511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:47.478130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78e4b1-0956-4afa-b41f-44cd3206c924 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:47.489680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1868] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:47.598125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:47.599622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.612281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc7195-7e08-4251-9dbe-1e70513889e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:47.621354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:47.634186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.634186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.634186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] PATH_INFO: `/v3/users/fb91a6abfb5746bb9b19f67f8b01fe3a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.635442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb91a6abfb5746bb9b19f67f8b01fe3a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:47.637457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:47.708055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb91a6abfb5746bb9b19f67f8b01fe3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.708773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb91a6abfb5746bb9b19f67f8b01fe3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.709542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb91a6abfb5746bb9b19f67f8b01fe3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.724340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:47.732616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb91a6abfb5746bb9b19f67f8b01fe3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.743259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb91a6abfb5746bb9b19f67f8b01fe3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:47.757482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1869] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:47 2022] DELETE /identity/v3/users/fb91a6abfb5746bb9b19f67f8b01fe3a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:47.762254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.762801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.763087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.788001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:47.789798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.861608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1870] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:47.870985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:47.872735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.932120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472a59-36f9-44d0-b066-0bb0deb0f75a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:47.935796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:47.936485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.936771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.937051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.943484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:47.945473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:47.947124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:47.948681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:47.959373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1871] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:47.960202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca238f2-9ac9-49b1-9da4-eef78eee9d20 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:47.966525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:47.967389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:47.967753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:47.967943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] PATH_INFO: `/v3/projects/2527a2eedd2d4c0bb0ac206456cac688` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:47.970704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2527a2eedd2d4c0bb0ac206456cac688)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:47.973215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.028214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2527a2eedd2d4c0bb0ac206456cac688', 'request_id': 'req-bc548200-25d6-4ed5-bf11-54b76af2e0a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.035888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2527a2eedd2d4c0bb0ac206456cac688', 'request_id': 'req-bc548200-25d6-4ed5-bf11-54b76af2e0a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.042646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2527a2eedd2d4c0bb0ac206456cac688', 'request_id': 'req-bc548200-25d6-4ed5-bf11-54b76af2e0a0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.057585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] The token cache is being invalidate because project 2527a2eedd2d4c0bb0ac206456cac688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:48.057883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.058547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.060276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1872] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:47 2022] DELETE /identity/v3/projects/2527a2eedd2d4c0bb0ac206456cac688 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:48.064373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.074618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7248e0-3d8b-4447-8310-e251c474fa1b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.141702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.142710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.143062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.143634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] PATH_INFO: `/v3/users/4f29601df0a14b05ac790adc56c26501` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.145497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f29601df0a14b05ac790adc56c26501)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.147546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.205163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f29601df0a14b05ac790adc56c26501'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.205968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f29601df0a14b05ac790adc56c26501'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.206756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f29601df0a14b05ac790adc56c26501'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.219709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f29601df0a14b05ac790adc56c26501'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.231410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f29601df0a14b05ac790adc56c26501'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.237797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:48.248445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1873] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:48 2022] DELETE /identity/v3/users/4f29601df0a14b05ac790adc56c26501 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:48.256685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.258802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.272682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548200-25d6-4ed5-bf11-54b76af2e0a0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.347327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:48.368834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.371222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.372575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.374485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.376367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.376935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] PATH_INFO: `/v3/projects/79d68780ab0540febd8d9a35cc8db7b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.378689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d68780ab0540febd8d9a35cc8db7b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.381034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.387582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dda9e-a4fb-4d44-96b2-db07b723fbd2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.392670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.393568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.393896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.394176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] PATH_INFO: `/v3/users/f6cad538f7c04e4987a87242b81bc0ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.395486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6cad538f7c04e4987a87242b81bc0ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.397470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.462535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6cad538f7c04e4987a87242b81bc0ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.464200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6cad538f7c04e4987a87242b81bc0ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.464862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6cad538f7c04e4987a87242b81bc0ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.465544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d68780ab0540febd8d9a35cc8db7b6', 'request_id': 'req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.475815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d68780ab0540febd8d9a35cc8db7b6', 'request_id': 'req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.484970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6cad538f7c04e4987a87242b81bc0ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.491454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d68780ab0540febd8d9a35cc8db7b6', 'request_id': 'req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.495394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-31a7cf06-9989-4a91-9e06-611e2dbd9032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6cad538f7c04e4987a87242b81bc0ae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.514808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1874] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:48 2022] DELETE /identity/v3/users/f6cad538f7c04e4987a87242b81bc0ae => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:48.519489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.519773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] The token cache is being invalidate because project 79d68780ab0540febd8d9a35cc8db7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:48.520087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.520338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.520814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.520925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.523350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1875] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:48 2022] DELETE /identity/v3/projects/79d68780ab0540febd8d9a35cc8db7b6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:48.527475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.528922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.548207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:48.551126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.646433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b50f3-8bae-4ac9-9aac-c0df84bffcf3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.653268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.654213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.654862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.654862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] PATH_INFO: `/v3/users/e66c636ab37e43f5aa3d887b98fbc331` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.657667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e66c636ab37e43f5aa3d887b98fbc331)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.659903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.660383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1876] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:48.666581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.669375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.747396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db173dd-0bfd-4bf7-9e14-76282338f484 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.752594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.753639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.754087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.754497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.765612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.767200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.780967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1877] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:48.781649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e66c636ab37e43f5aa3d887b98fbc331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.781803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:48.782353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e66c636ab37e43f5aa3d887b98fbc331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.783365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:48.801392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06d71-5a77-4db9-8bdc-d6c5bbcb5b39 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:48.808024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e66c636ab37e43f5aa3d887b98fbc331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.808651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:48.809748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.810172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.810656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] PATH_INFO: `/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:48.812030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd8ff34c4f8457b9c1fcfbfe060c4f2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:48.814511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e66c636ab37e43f5aa3d887b98fbc331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.816214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:48.824352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0af47320-9167-4fba-bd1e-f09e5c16e4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e66c636ab37e43f5aa3d887b98fbc331'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.829753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:48.830110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:48.851491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1878] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:48 2022] DELETE /identity/v3/users/e66c636ab37e43f5aa3d887b98fbc331 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:48.922514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd8ff34c4f8457b9c1fcfbfe060c4f2', 'request_id': 'req-9af59441-313a-4736-b118-5d036b2f2bb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.935794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd8ff34c4f8457b9c1fcfbfe060c4f2', 'request_id': 'req-9af59441-313a-4736-b118-5d036b2f2bb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.949870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd8ff34c4f8457b9c1fcfbfe060c4f2', 'request_id': 'req-9af59441-313a-4736-b118-5d036b2f2bb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.980768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] The token cache is being invalidate because project 9bd8ff34c4f8457b9c1fcfbfe060c4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:48.981368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.982250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9af59441-313a-4736-b118-5d036b2f2bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:48.984069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1879] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:48 2022] DELETE /identity/v3/projects/9bd8ff34c4f8457b9c1fcfbfe060c4f2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:48.989884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:48.990958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:48.991439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.014391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:49.015969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.095126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1880] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:49.104092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.104401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.105118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.128266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:49.130377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.153995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.162068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1881] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:49.168898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cf9e4-c560-4be2-ab66-c2328c12b702 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.180045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.180850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.181141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.181468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.182801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:49.183155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.185319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.208515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.219645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1882] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:49.220393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fdd89-24a2-42c8-b934-fdcc018d109b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.226193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.226938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.227339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.227529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.228505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.231810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.269023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1883] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:49 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:49.270739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.281195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94eb8-9cf7-4355-8832-20e523c37f42 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.345545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.346228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.346513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.346930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.347800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.349930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.373077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1884] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:49 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:49.377732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.390275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927cc88-4e6a-4636-a26e-4f1be901ab6f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.405243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.407071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.412623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.415113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.415407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.415690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.419617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.421189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e69d3-a05d-4380-b530-ef71e2d35086 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.421866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.426712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.427713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.428100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.428593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] PATH_INFO: `/v3/projects/cf4b57c59b954c97915e54d3a7b80a68` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.430165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4b57c59b954c97915e54d3a7b80a68)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.432388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.449805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1885] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:49.452770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.455530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.472946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1e80a-bb34-4e83-91e0-1bcab9ffc02b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.479983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.481420 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.481747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.482303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] PATH_INFO: `/v3/projects/b532dada1c104aeeb996130e013431b1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.485769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b532dada1c104aeeb996130e013431b1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.487853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.491163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4b57c59b954c97915e54d3a7b80a68', 'request_id': 'req-1a8ff8ba-2f31-434b-998e-384718e7c222'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.498662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4b57c59b954c97915e54d3a7b80a68', 'request_id': 'req-1a8ff8ba-2f31-434b-998e-384718e7c222'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.507506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4b57c59b954c97915e54d3a7b80a68', 'request_id': 'req-1a8ff8ba-2f31-434b-998e-384718e7c222'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.511085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:49.526323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] The token cache is being invalidate because project cf4b57c59b954c97915e54d3a7b80a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:49.526638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.527460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.529713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1886] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:49 2022] DELETE /identity/v3/projects/cf4b57c59b954c97915e54d3a7b80a68 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:49.530728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.533230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.548933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b532dada1c104aeeb996130e013431b1', 'request_id': 'req-cf19ea6c-0377-4b73-adba-16e9fb4aa778'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.556610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b532dada1c104aeeb996130e013431b1', 'request_id': 'req-cf19ea6c-0377-4b73-adba-16e9fb4aa778'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.568036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b532dada1c104aeeb996130e013431b1', 'request_id': 'req-cf19ea6c-0377-4b73-adba-16e9fb4aa778'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.588188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] The token cache is being invalidate because project b532dada1c104aeeb996130e013431b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:49.588959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.590142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf19ea6c-0377-4b73-adba-16e9fb4aa778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.592347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1887] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:49 2022] DELETE /identity/v3/projects/b532dada1c104aeeb996130e013431b1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:49.596359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.596813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.597265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.631703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:49.633732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.646430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff8ba-2f31-434b-998e-384718e7c222 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.679954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:49.733432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.734962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.735420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.735783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.736645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.739639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1888] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:49.739843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9715b-8951-44f0-94b6-8fb9cc5008fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.741120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.743891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.759323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1889] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:49.760774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.761141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.761474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.771277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cb705-6b1c-467d-8f35-aec166ab4d41 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.777446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.778451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.778815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.779170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] PATH_INFO: `/v3/users/cf5eedfd6a3e4e78a65391c399a9db19` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.782024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5eedfd6a3e4e78a65391c399a9db19)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:49.784757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:49.864383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5eedfd6a3e4e78a65391c399a9db19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.865251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5eedfd6a3e4e78a65391c399a9db19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.865799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5eedfd6a3e4e78a65391c399a9db19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.882543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5eedfd6a3e4e78a65391c399a9db19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.892454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5eedfd6a3e4e78a65391c399a9db19'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:49.906174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1890] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:49 2022] DELETE /identity/v3/users/cf5eedfd6a3e4e78a65391c399a9db19 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:49.907084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:49.908471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.918885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:49.920106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:49.984863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8d8c8-b43d-4be7-9528-318ef525c705 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:49.990067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:49.991097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:49.991692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:49.992049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] PATH_INFO: `/v3/projects/740d1155ea774f03880d2b3ce2278874/users/61cae71e698f4e08863f6deac5ea1351/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:49.999655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1891] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:50.004642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.015556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e72223-972b-4b88-91a9-3bcfdf3426b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.015690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=740d1155ea774f03880d2b3ce2278874, user_id=61cae71e698f4e08863f6deac5ea1351, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.018487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.020886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.021555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.021963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.022297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.023157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:50.023867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.026122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449977c-12f0-4ab0-a6e5-bcf84fa4ebc8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.037782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1892] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:49 2022] PUT /identity/v3/projects/740d1155ea774f03880d2b3ce2278874/users/61cae71e698f4e08863f6deac5ea1351/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:50.042476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1893] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:50.048741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.050921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.051488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.051876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.089974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:50.111865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] MFA Rules not processed for user `61cae71e698f4e08863f6deac5ea1351`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:50.113259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.130573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27e222-850a-4d87-a4cd-286ac2abea4a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.131395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:50.141560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.142612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.142970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.143614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.144630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.150654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.193986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1894] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:50.195616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.221762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1895] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:50.234330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.239114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.294208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fa30-d509-44ae-9da7-fd8347dc8ebc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.299515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.300981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.301283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.301705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.302419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.304436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.316489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc48120-6532-434d-b923-21b1d1ac9292 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.329500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1896] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:50.331213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.333699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.349850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185ac3-ae0b-4168-b40c-3d3ec8b237fb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.356212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.357187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.357551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.357898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] PATH_INFO: `/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.359245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7089c93f7940c1930f31a1f2b0dc30)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.361307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.380981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.381744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.382072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.382384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.421555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7089c93f7940c1930f31a1f2b0dc30', 'request_id': 'req-f95af774-e770-4bab-8dab-6b5c9097a9df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:50.427977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7089c93f7940c1930f31a1f2b0dc30', 'request_id': 'req-f95af774-e770-4bab-8dab-6b5c9097a9df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:50.440573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7089c93f7940c1930f31a1f2b0dc30', 'request_id': 'req-f95af774-e770-4bab-8dab-6b5c9097a9df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:50.453628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] The token cache is being invalidate because project ce7089c93f7940c1930f31a1f2b0dc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:50.453924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:50.454703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:50.454866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.456440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1897] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:50 2022] DELETE /identity/v3/projects/ce7089c93f7940c1930f31a1f2b0dc30 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:50.457297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58481b86-571c-44cd-8b2e-61a25001a58e service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.457416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.458683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.467682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1898] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:50.469090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9d23-74bc-4fd3-991e-ed87f3ffcdd4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.469379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9d23-74bc-4fd3-991e-ed87f3ffcdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.469655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff9d23-74bc-4fd3-991e-ed87f3ffcdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.490276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1eff9d23-74bc-4fd3-991e-ed87f3ffcdd4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:50.491732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1eff9d23-74bc-4fd3-991e-ed87f3ffcdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.495987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:50.513997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:50.535762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95af774-e770-4bab-8dab-6b5c9097a9df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.541216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.542057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.543084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.543371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.544112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.545875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.568603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1899] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:50.570858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.587515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8dccc-ad40-4c79-aa91-a39e8e0edb23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.601945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1900] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:50.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.610860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.618209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.620459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.620761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.621045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.623975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.626362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.643713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1901] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:50.647770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.660803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f219b-a1ec-4244-aba3-74c08a92f9cd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.663610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:50.665277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.666887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.667865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.668436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.668910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] PATH_INFO: `/v3/projects/7dba43132e1a4f7daacfbd7a5630d0ab/users/122c99e1bb394f91a66db1a8697d3026/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.679832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1902] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:50.686610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dba43132e1a4f7daacfbd7a5630d0ab, user_id=122c99e1bb394f91a66db1a8697d3026, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.689748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a44aa3-e722-4833-85c4-9a60fac8d082 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.712452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.724177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1903] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:50 2022] PUT /identity/v3/projects/7dba43132e1a4f7daacfbd7a5630d0ab/users/122c99e1bb394f91a66db1a8697d3026/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:50.735534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.735828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.736115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.754686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae4a56-f41d-417f-b7bd-cdbd38c5f397 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.792839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] MFA Rules not processed for user `122c99e1bb394f91a66db1a8697d3026`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:50.794760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:50.840962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.841895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.842314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.842737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.844195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:50.844689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.847889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.870722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1904] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:50.874728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.889572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bb054-e13a-4f0b-b582-537e1692ba7b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:50.896496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:50.897387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:50.898544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:50.898913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:50.900034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:50.902762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:50.960034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1905] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:50.963031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.967986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1906] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:50.969078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:50.991402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60b14f-2257-4e67-ad0b-e4a038731456 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.088485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920ddf0-adae-45ed-86fa-224893c488f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.119815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.119815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.119815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.120349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.120831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.123151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.140951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.141635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.141916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.142194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.142898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.145344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.160816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1907] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:51.162245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.175215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7376-aac0-4b7a-ad67-f54b25f6d137 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.180717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.181570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.181745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.182055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.182793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.184974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.200479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1908] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:50 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:51.201168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1909] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:51.201280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.203535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.210667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:51.215133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda6fb-1a89-49ca-af52-1c69fbb44075 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.220809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:51.297106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792bf35-2314-46a6-bba4-e5eb269e071c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.300365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.301304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.301646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.301872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.302184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.302608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.302854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.303151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.303412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.304103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.304202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.306090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.315126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1910] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:51.319458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.329502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1911] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:51 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:51.332891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86b20-98c5-4716-9b8d-ec5e2fc3593b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.333410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.337607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.338508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.338794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.339071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] PATH_INFO: `/v3/projects/12d74028509b4db4912012f8c1d67416/users/0e9a16663524460097aba27a45bf1933/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.343951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fc6ee-ff38-49b7-baa7-33297a3d008e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.349145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.349819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.350111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.350381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.351179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.353132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.363365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d74028509b4db4912012f8c1d67416, user_id=0e9a16663524460097aba27a45bf1933, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.367351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1912] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:51.367473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f839916-64ec-4115-9c9c-fd4dae25f446 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.368279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.382224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a5ed-d0ba-4ece-9622-9113d7bb7bca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.383384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1913] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:51 2022] PUT /identity/v3/projects/12d74028509b4db4912012f8c1d67416/users/0e9a16663524460097aba27a45bf1933/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:51.394349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b657ab3-ea07-424b-997c-10762558910d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.394656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b657ab3-ea07-424b-997c-10762558910d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.394913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b657ab3-ea07-424b-997c-10762558910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.422355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b657ab3-ea07-424b-997c-10762558910d None None] MFA Rules not processed for user `0e9a16663524460097aba27a45bf1933`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:51.423641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b657ab3-ea07-424b-997c-10762558910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:51.457215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.458066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.458349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.460193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] PATH_INFO: `/v3/projects/cee6e37a228047629add9874cfb0a551/users/9818950c5fbf41fabc9543d60023242c/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.481062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee6e37a228047629add9874cfb0a551, user_id=9818950c5fbf41fabc9543d60023242c, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.483873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.501978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1914] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:51 2022] PUT /identity/v3/projects/cee6e37a228047629add9874cfb0a551/users/9818950c5fbf41fabc9543d60023242c/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:51.507894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.513196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1915] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:51.517715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b657ab3-ea07-424b-997c-10762558910d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.532119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b657ab3-ea07-424b-997c-10762558910d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.588671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7347f8-9ba0-4c05-8e95-f74c9f31e25e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.594654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.595348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.595641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.595936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.597654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.599292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.606797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.607244 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.607607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.607950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.608715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.610778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.611942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1916] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:51.616513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.623894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:51.634708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:51.634830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c8610-4206-4ff0-9bae-d33cbf33d120 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.640693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.641655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.644977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.644977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] PATH_INFO: `/v3/projects/cee6e37a228047629add9874cfb0a551/users/9818950c5fbf41fabc9543d60023242c/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.644977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee6e37a228047629add9874cfb0a551, user_id=9818950c5fbf41fabc9543d60023242c, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.645288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1917] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:51 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:51.646581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.647136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f02d5-18ee-4736-8fb1-fa178834b825 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.659642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1918] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:51 2022] PUT /identity/v3/projects/cee6e37a228047629add9874cfb0a551/users/9818950c5fbf41fabc9543d60023242c/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:51.670109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.670399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.670676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.715293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] MFA Rules not processed for user `9818950c5fbf41fabc9543d60023242c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:51.717425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:51.741659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc0b9-82ef-43f1-98a2-09641a6d090e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.806607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1919] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:51.818778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.820138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:51.821710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.822477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.822833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.823200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.824143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.827401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.863811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1920] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:51 2022] POST /identity/v3/users => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:51.865272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.877612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0031939-a742-4ec8-a5f3-2b884bb8d968 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.901035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.911962 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.912476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.913221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.914034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.915699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:51.936094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1921] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:51.941646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:51.948091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c843fd-d3fa-4d9f-81c0-95d7f5ce5cd2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.956295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.957547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.957927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.958654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.966340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924d497-ad70-44aa-a9a6-836e08d19660 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:51.984633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:51.985890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:51.986191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:51.986480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] PATH_INFO: `/v3/projects/0c52a52cfcb34a26bb1fc9aede249639/users/cd2e2132dadf4e3da851ffa49a2ff599/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:51.990444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:51.993459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.010436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1922] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:52.034287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c52a52cfcb34a26bb1fc9aede249639, user_id=cd2e2132dadf4e3da851ffa49a2ff599, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.037716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.059111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1923] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:51 2022] PUT /identity/v3/projects/0c52a52cfcb34a26bb1fc9aede249639/users/cd2e2132dadf4e3da851ffa49a2ff599/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:52.059391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:52.066664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:52.069134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:52.070266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210712-2bb9-45b6-9eea-417d2644d0d2 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:52.139305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028fa02-6853-45b8-b823-dc1c56cea099 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:52.143172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:52.144087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.144491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.144871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.153007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:52.154036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.154500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.154982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.155830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.157817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.172459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1924] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:52.175422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:52.189757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2da08-2661-42b6-9a80-87ae4cb9bca5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:52.192762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:52.195616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:52.196680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.197068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.197446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] PATH_INFO: `/v3/projects/0c52a52cfcb34a26bb1fc9aede249639/users/cd2e2132dadf4e3da851ffa49a2ff599/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.199697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c52a52cfcb34a26bb1fc9aede249639, user_id=cd2e2132dadf4e3da851ffa49a2ff599, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.214222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87873d-95d3-45ef-a041-eff049c15509 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.245096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1925] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:52 2022] PUT /identity/v3/projects/0c52a52cfcb34a26bb1fc9aede249639/users/cd2e2132dadf4e3da851ffa49a2ff599/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:52.256601 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.258792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.259523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.262114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.263695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.269769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:52.275430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1926] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:52.295209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] MFA Rules not processed for user `cd2e2132dadf4e3da851ffa49a2ff599`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:52.296820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:52.439824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1927] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:52.454681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:52.467024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca8b77-cdef-40bf-bc39-69175f15865b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:52.537803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:52.538450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.538738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.539012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.555320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.555320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d62e7-5170-4806-b79e-4d7f4bfc537f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:52.587454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1928] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:52.811970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:52.910445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ce138-eb68-4f89-9daa-3d47fc11b715 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:52.922852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:52.924980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:52.925462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:52.925907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] PATH_INFO: `/v3/users/0e9a16663524460097aba27a45bf1933` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:52.927469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9a16663524460097aba27a45bf1933)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:52.930719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:53.010382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9a16663524460097aba27a45bf1933'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.011198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9a16663524460097aba27a45bf1933'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.011941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9a16663524460097aba27a45bf1933'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.042062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9a16663524460097aba27a45bf1933'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.054790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9a16663524460097aba27a45bf1933'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.081710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1929] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:52 2022] DELETE /identity/v3/users/0e9a16663524460097aba27a45bf1933 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:53.086338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:53.086722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:53.087120 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:53.105303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:53.106663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:53.193362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1930] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:53.203040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:53.205205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:53.216095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:53.293383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce686-6f2f-447e-9cef-0aacbb1bebf8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:53.297004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:53.297665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:53.297957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:53.298241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:53.307679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:53.309182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:53.319096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1931] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:53.670259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:53.671710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:53.696291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404e8a-311d-4744-bfc6-d64b33a03270 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:53.705901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:53.707118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:53.707397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:53.707821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] PATH_INFO: `/v3/projects/12d74028509b4db4912012f8c1d67416` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:53.719922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d74028509b4db4912012f8c1d67416)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:53.722116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:53.787879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d74028509b4db4912012f8c1d67416', 'request_id': 'req-f3602bbc-097e-4755-9bfe-b9691cea57c8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.790089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:53.802607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d74028509b4db4912012f8c1d67416', 'request_id': 'req-f3602bbc-097e-4755-9bfe-b9691cea57c8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.821839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d74028509b4db4912012f8c1d67416', 'request_id': 'req-f3602bbc-097e-4755-9bfe-b9691cea57c8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.843303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] The token cache is being invalidate because project 12d74028509b4db4912012f8c1d67416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:53.843779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.844617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:53.846561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1932] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:53 2022] DELETE /identity/v3/projects/12d74028509b4db4912012f8c1d67416 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:53.854400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:53.877858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004026a4-e90b-44ee-8bdd-9881a106cd28 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:53.970851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:53.974611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:53.974611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:53.974611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] PATH_INFO: `/v3/users/cd2e2132dadf4e3da851ffa49a2ff599` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:53.981598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2e2132dadf4e3da851ffa49a2ff599)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:53.985200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:54.059474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2e2132dadf4e3da851ffa49a2ff599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:54.060808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2e2132dadf4e3da851ffa49a2ff599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:54.061458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2e2132dadf4e3da851ffa49a2ff599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:54.090394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2e2132dadf4e3da851ffa49a2ff599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:54.104015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b18c0c10-4cc0-492a-8bbd-c23547aeae7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2e2132dadf4e3da851ffa49a2ff599'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:54.131629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1933] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:53 2022] DELETE /identity/v3/users/cd2e2132dadf4e3da851ffa49a2ff599 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:54.926643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:54.929411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:54.967960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3602bbc-097e-4755-9bfe-b9691cea57c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.101077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.104523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.104969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.105471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] PATH_INFO: `/v3/projects/0c52a52cfcb34a26bb1fc9aede249639` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.106901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c52a52cfcb34a26bb1fc9aede249639)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.109890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.226233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c52a52cfcb34a26bb1fc9aede249639', 'request_id': 'req-214f658b-42ad-493a-a3b3-c0c122225a87'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:55.232663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c52a52cfcb34a26bb1fc9aede249639', 'request_id': 'req-214f658b-42ad-493a-a3b3-c0c122225a87'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:55.241470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c52a52cfcb34a26bb1fc9aede249639', 'request_id': 'req-214f658b-42ad-493a-a3b3-c0c122225a87'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:55.260829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] The token cache is being invalidate because project 0c52a52cfcb34a26bb1fc9aede249639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:55.261339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:55.262061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-214f658b-42ad-493a-a3b3-c0c122225a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:55.264466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1934] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:54 2022] DELETE /identity/v3/projects/0c52a52cfcb34a26bb1fc9aede249639 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:55.274067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.274561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.274653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.294212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:55.295459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:55.331117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:55.388800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1935] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:55.390793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.391342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.391811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.422204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:55.423546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:55.437092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1936] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:55.441876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.452888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4910a-1b28-4c1f-be37-45ae61fb46c2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.463650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.464473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.464815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.465105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.465894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:55.466709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.467903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.488881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1937] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:55.493145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.503289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e03e7-5324-485f-8c9f-bd27d62c82ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.508599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.509284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.509571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.509849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.510563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.512533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.562549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1938] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:55 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:55.564199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.576766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aff079-7b30-40bd-a82b-d33e9bc40379 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.679142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.681071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.681633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.682170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.683022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.685308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.714303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1939] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:55 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:55.718979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.731670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db1ff0-f81e-4085-a4c0-428b17ec8c50 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.737028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.737889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.738269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.738641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.739476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.741265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.759291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1940] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:09:55.763467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.781615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7198293-ae8f-4ccf-b25e-9d4e2e1a4d73 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.789483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.790079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.790735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.791063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.791814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.794243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ab975-2733-4598-a99c-0a7cc31f3e17 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.813396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1941] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:55.815830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:55.831305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a5c92-4536-46b4-b894-ca219d48094e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:55.838192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:55.839294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.839724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.840134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] PATH_INFO: `/v3/projects/9e8df6d266cc4f929f6fe38b84ac6044/users/cf6be20944a8402bbc61f8fd208947f1/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.863627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8df6d266cc4f929f6fe38b84ac6044, user_id=cf6be20944a8402bbc61f8fd208947f1, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:55.866395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:55.900480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1942] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:55 2022] PUT /identity/v3/projects/9e8df6d266cc4f929f6fe38b84ac6044/users/cf6be20944a8402bbc61f8fd208947f1/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:55.906069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:55.906363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:55.906648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:55.955745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] MFA Rules not processed for user `cf6be20944a8402bbc61f8fd208947f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:55.957071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:56.054786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1943] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:56.059607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.067230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:56.148070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48f67f-dde0-4cc1-a646-9728fc73cd73 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.153414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.154121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.154443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.154829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.155682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.157669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.187696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1944] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:56 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:56.191740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.205793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aa342-512f-40f8-b7a2-182a3723c31d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.231542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:56.266906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.268980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:56.299311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.300872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.310934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.311347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.313316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.317077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.360672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1945] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:56 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:56.365983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.383015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7710b-d863-4077-8d25-db6573a46e81 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.397527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.398586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.399930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.401007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.401774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.404498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.408380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574138-82ca-4698-801b-3ed9429f8641 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.421384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.423380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.423380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.423380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] PATH_INFO: `/v3/users/122c99e1bb394f91a66db1a8697d3026` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.424688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122c99e1bb394f91a66db1a8697d3026)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.432837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1946] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:56.433747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.434896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.448370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19682-30aa-4d4b-8df2-24e9f01dafed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.461401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.462473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.462876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.463356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] PATH_INFO: `/v3/projects/c05a65fc6ed9434bb3148eb90e67d147/users/3e1e8d50f00b4b77b2731b5d79acbcbe/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.516229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05a65fc6ed9434bb3148eb90e67d147, user_id=3e1e8d50f00b4b77b2731b5d79acbcbe, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.522598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.531911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122c99e1bb394f91a66db1a8697d3026'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:56.532664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122c99e1bb394f91a66db1a8697d3026'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:56.559796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122c99e1bb394f91a66db1a8697d3026'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:56.560599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122c99e1bb394f91a66db1a8697d3026'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:56.575220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1947] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:56 2022] PUT /identity/v3/projects/c05a65fc6ed9434bb3148eb90e67d147/users/3e1e8d50f00b4b77b2731b5d79acbcbe/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:56.579397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.579936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b84eff-a2df-4592-ba10-e8c78c4c5097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122c99e1bb394f91a66db1a8697d3026'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:56.586763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:56.604568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1948] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:56 2022] DELETE /identity/v3/users/122c99e1bb394f91a66db1a8697d3026 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:56.610081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.610383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.610670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.649378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:56.650669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:56.662027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:56.717569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43f7a4-017c-4f8c-b14c-0585128c5f58 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.784645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1949] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:56.794610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.796849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:56.810533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.811245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.811529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.811804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.812578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.814915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.841629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1950] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:56.845797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:56.875405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0cf63-9625-4337-9cae-6919f2a80809 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.884092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:56.884949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.885237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.885527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] PATH_INFO: `/v3/projects/c05a65fc6ed9434bb3148eb90e67d147/users/3e1e8d50f00b4b77b2731b5d79acbcbe/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.888010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05a65fc6ed9434bb3148eb90e67d147, user_id=3e1e8d50f00b4b77b2731b5d79acbcbe, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:56.890800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9334b2-d790-4edf-bcff-cc034a72c640 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:56.921298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dc1e7-7f26-42ab-a2ff-2cad76bfcbc8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:56.921540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1951] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:56 2022] PUT /identity/v3/projects/c05a65fc6ed9434bb3148eb90e67d147/users/3e1e8d50f00b4b77b2731b5d79acbcbe/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:09:56.931948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:56.932342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:56.932802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:56.954444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:56.975785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] MFA Rules not processed for user `3e1e8d50f00b4b77b2731b5d79acbcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:56.975785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:57.004280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:57.004884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:57.005213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:57.005464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:57.073827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1952] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:57.088259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:57.089905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:57.104534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:57.110109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:57.122934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1953] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:09:57.182526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a568ee1-51e5-41c9-96c2-2393faff63e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:57.186369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:57.187138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:57.187514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:57.187892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:57.199187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:57.200814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:57.220866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1954] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:57.303266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:57.312817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947dd20-c034-43fe-81ba-b97fcd3e83ab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:57.316609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:57.317263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:57.317602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:57.318007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:57.319829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:57.334013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:57.394735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:57.396341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:57.407177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1955] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:09:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:09:57.817911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:57.819719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:57.844597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa9b8-0be5-42ad-9832-35be4593575b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:57.860280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:57.861886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:57.862227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:57.862514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] PATH_INFO: `/v3/projects/7dba43132e1a4f7daacfbd7a5630d0ab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:57.863847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dba43132e1a4f7daacfbd7a5630d0ab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:57.866003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:57.924253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dba43132e1a4f7daacfbd7a5630d0ab', 'request_id': 'req-c9566f62-dd3e-403d-8432-61fd99d92ade'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:57.932349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dba43132e1a4f7daacfbd7a5630d0ab', 'request_id': 'req-c9566f62-dd3e-403d-8432-61fd99d92ade'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:57.942566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dba43132e1a4f7daacfbd7a5630d0ab', 'request_id': 'req-c9566f62-dd3e-403d-8432-61fd99d92ade'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:57.982108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] The token cache is being invalidate because project 7dba43132e1a4f7daacfbd7a5630d0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:57.983299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:57.984660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:57.987889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1956] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:57 2022] DELETE /identity/v3/projects/7dba43132e1a4f7daacfbd7a5630d0ab => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:57.992269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:58.013597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9cac6-e4df-4bba-8715-033020413e21 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:58.072701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:58.074073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:58.074073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:58.074311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] PATH_INFO: `/v3/users/9818950c5fbf41fabc9543d60023242c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:58.075342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9818950c5fbf41fabc9543d60023242c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:58.077322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:58.129926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9818950c5fbf41fabc9543d60023242c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.130617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9818950c5fbf41fabc9543d60023242c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.143763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9818950c5fbf41fabc9543d60023242c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.144474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9818950c5fbf41fabc9543d60023242c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.157915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cc91ee94-719f-4f6f-b423-b1a89096e668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9818950c5fbf41fabc9543d60023242c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.181184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1957] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:09:57 2022] DELETE /identity/v3/users/9818950c5fbf41fabc9543d60023242c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:58.542691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:58.544267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:58.611290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9566f62-dd3e-403d-8432-61fd99d92ade admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:58.666989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:58.735164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:58.736754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:58.737552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:58.738332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] PATH_INFO: `/v3/projects/cee6e37a228047629add9874cfb0a551` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:58.744925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee6e37a228047629add9874cfb0a551)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:58.748588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:58.829823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee6e37a228047629add9874cfb0a551', 'request_id': 'req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.841585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee6e37a228047629add9874cfb0a551', 'request_id': 'req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.856302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee6e37a228047629add9874cfb0a551', 'request_id': 'req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.889496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] The token cache is being invalidate because project cee6e37a228047629add9874cfb0a551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:09:58.889804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.891223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2201a0ff-8b9b-47ce-a0d8-ca563f7cf9a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:09:58.893717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1958] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:09:58 2022] DELETE /identity/v3/projects/cee6e37a228047629add9874cfb0a551 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:58.898783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:58.899083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:58.899433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:58.929902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:58.931588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:58.976139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:59.031720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1959] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:09:59.040813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.041104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.041383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.062630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:59.063855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:59.076403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1960] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:59.080865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.095922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40336f12-29f9-4130-9619-bef2f52ff19f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.102423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.103320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.103785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.104250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.105165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:09:59.105674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.107687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.128611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1961] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:09:59.133100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.144802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc344b5-9cd6-464f-b19b-1e3b07bafcdf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.157226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.157907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.158360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.158721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.160290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.162031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.200786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1962] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:59.207687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.223616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556856-5319-46b5-8123-30fc00774c17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.304840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.305698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.305999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.306263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.306984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.308941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.334020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1963] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:09:59.338200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.375150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4183e-7514-42d2-82d6-52d876734a53 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.381054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.382118 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.382410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.383063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.383804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.385451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.404442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1964] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:59.405898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.419393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f6f4f-73af-4b5b-8511-b9559a7c9676 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.430377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.431337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.431779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.432273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.433390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.436050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38521e46-4c25-43a4-97c6-f7560cca8c15 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.462491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1965] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:59.466851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.477390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412834-1ffe-4239-9e1d-6e2045b5365e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.483455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.488439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.488439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.488439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] PATH_INFO: `/v3/projects/9b4c8e691b724d86b21c6a30e5371dfb/users/6b929d7a2dc8473980dbe09faa484acd/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.503164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4c8e691b724d86b21c6a30e5371dfb, user_id=6b929d7a2dc8473980dbe09faa484acd, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.505969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.535155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1966] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:59 2022] PUT /identity/v3/projects/9b4c8e691b724d86b21c6a30e5371dfb/users/6b929d7a2dc8473980dbe09faa484acd/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:09:59.545555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.545847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.546125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.572717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] MFA Rules not processed for user `6b929d7a2dc8473980dbe09faa484acd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:09:59.573936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:09:59.652368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1967] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:09:59.656927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.725214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:59.741915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc018a1-991f-4778-bdfd-942ff37b1d1a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.746983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.748033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.748318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.751542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.752385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.754733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.785566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1968] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/projects => generated 399 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:59.786796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.798638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c7f06-9fa6-4338-ad6b-a6656df9b054 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.830982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:09:59.886503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.887208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.887690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.887971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.888719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.892606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.915346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1969] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:09:59 2022] POST /identity/v3/users => generated 306 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:09:59.919249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.938780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4e0fd-7e46-4cdd-9f65-208201e1b237 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.943683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.944502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.944798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.945079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.945812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:09:59.947437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:09:59.958531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1970] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:09:59.963045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:09:59.973346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bc00c-c33a-4ca5-a848-6fd531f3b959 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:09:59.978190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:09:59.979095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:09:59.979387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:09:59.979665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] PATH_INFO: `/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e/users/a5d14b2bfc2b44f5bf0b204fe04016e5/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:09:59.998775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83a51a1b3dd42a3b32021872ce77f9e, user_id=a5d14b2bfc2b44f5bf0b204fe04016e5, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:00.002115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:00.022268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1971] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:09:59 2022] PUT /identity/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e/users/a5d14b2bfc2b44f5bf0b204fe04016e5/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:00.023897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:00.035974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd6d72-8b14-4c08-9a11-bdec54702c6c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:00.108943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:00.109611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:00.109902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:00.110182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:00.110919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:00.112752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b57162-adc7-4a73-b2b6-20dc433d163c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:00.127639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1972] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:00.128206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:00.139464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9580533-1ae4-45d4-bb66-3a5c439581eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:00.144323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:00.145315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:00.145985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:00.145985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] PATH_INFO: `/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e/users/a5d14b2bfc2b44f5bf0b204fe04016e5/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:00.147905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83a51a1b3dd42a3b32021872ce77f9e, user_id=a5d14b2bfc2b44f5bf0b204fe04016e5, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:00.150607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:00.161805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1973] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:00 2022] PUT /identity/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e/users/a5d14b2bfc2b44f5bf0b204fe04016e5/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:00.172296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:00.172653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:00.172934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:00.202235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] MFA Rules not processed for user `a5d14b2bfc2b44f5bf0b204fe04016e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:00.203721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:00.285127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/1974] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:00.317030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:00.319025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:00.409948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478400e-1427-4e70-b077-4c131edc73a0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:00.417928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:00.419287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:00.420133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:00.420659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:00.435305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:00.437712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:00.454007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1975] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:00.539364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:00.549364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81268a8-9639-40e7-8c43-27faa0a0a2d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:00.553702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:00.554368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:00.554661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:00.554941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:00.640533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:00.686646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:00.688174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0441272-aef1-4387-b785-13a220317921 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:00.700159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/1976] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:02.792654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:02.794149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:02.898013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958ae07-d418-4bf0-bc24-8ee846f6fe08 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:02.903011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:02.903902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:02.904190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:02.904489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] PATH_INFO: `/v3/users/cf6be20944a8402bbc61f8fd208947f1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:02.905710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6be20944a8402bbc61f8fd208947f1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:02.907658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:02.938380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:02.967212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6be20944a8402bbc61f8fd208947f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:02.967924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6be20944a8402bbc61f8fd208947f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:02.986070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6be20944a8402bbc61f8fd208947f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:02.987019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6be20944a8402bbc61f8fd208947f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.005397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6be20944a8402bbc61f8fd208947f1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.040047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1977] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:02 2022] DELETE /identity/v3/users/cf6be20944a8402bbc61f8fd208947f1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:03.044523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:03.044987 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:03.045064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:03.071106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:03.072380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:03.198393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/1978] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:03.206208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:03.207995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:03.312161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640a359-ef6b-49a1-a3fb-76aa9c441506 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:03.315911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:03.316610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:03.316905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:03.317221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:03.326845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:03.328481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:03.344714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1979] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:03.677747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:03.679206 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:03.694953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416210d-463c-435c-aa75-17fbaaf5f8a0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:03.700648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:03.701577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:03.701893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:03.702347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] PATH_INFO: `/v3/projects/9e8df6d266cc4f929f6fe38b84ac6044` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:03.704282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8df6d266cc4f929f6fe38b84ac6044)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:03.707092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:03.784254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8df6d266cc4f929f6fe38b84ac6044', 'request_id': 'req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.794290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8df6d266cc4f929f6fe38b84ac6044', 'request_id': 'req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.803610 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8df6d266cc4f929f6fe38b84ac6044', 'request_id': 'req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.815061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] The token cache is being invalidate because project 9e8df6d266cc4f929f6fe38b84ac6044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:03.815383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.816132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.817708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/1980] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:03 2022] DELETE /identity/v3/projects/9e8df6d266cc4f929f6fe38b84ac6044 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:03.821647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:03.831574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08937783-5a55-46f7-a3e9-528a9ec90067 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:03.891079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:03.891973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:03.892263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:03.892578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] PATH_INFO: `/v3/users/3e1e8d50f00b4b77b2731b5d79acbcbe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:03.894164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e1e8d50f00b4b77b2731b5d79acbcbe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:03.896197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:03.945635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e1e8d50f00b4b77b2731b5d79acbcbe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.946317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e1e8d50f00b4b77b2731b5d79acbcbe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.947743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:03.965378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e1e8d50f00b4b77b2731b5d79acbcbe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.966108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e1e8d50f00b4b77b2731b5d79acbcbe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.975079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b68ea846-36dd-4305-9a65-d99fa06c661b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e1e8d50f00b4b77b2731b5d79acbcbe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:03.990507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1981] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:03 2022] DELETE /identity/v3/users/3e1e8d50f00b4b77b2731b5d79acbcbe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:04.323760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.325595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:04.355680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f345-e083-41a7-b8b2-7c0fb7a7400e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.418223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.419108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.419396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.419674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] PATH_INFO: `/v3/projects/c05a65fc6ed9434bb3148eb90e67d147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.420941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05a65fc6ed9434bb3148eb90e67d147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.422986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.429909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:04.476669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05a65fc6ed9434bb3148eb90e67d147', 'request_id': 'req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:04.482356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05a65fc6ed9434bb3148eb90e67d147', 'request_id': 'req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:04.491166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05a65fc6ed9434bb3148eb90e67d147', 'request_id': 'req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:04.503002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] The token cache is being invalidate because project c05a65fc6ed9434bb3148eb90e67d147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:04.503317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:04.503954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e86db1d0-6bd5-4deb-b1d2-61f3e488f733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:04.505643 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/1982] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:04 2022] DELETE /identity/v3/projects/c05a65fc6ed9434bb3148eb90e67d147 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:04.511405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.511576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.511855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.532024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:04.533302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:04.609390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1983] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:04.609651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.609651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.609651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.633626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:04.634830 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:04.642714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/1984] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:04.647070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.657079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b61872-1bf9-4bbf-b34c-37f2b4df65b3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.662576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.663320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.663612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.663895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.664674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:04.664987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.666732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.680101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1985] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:04.684201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.693627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ce197-4a56-47a1-a8b2-4edc62077e5a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.698348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.699035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.699281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.699613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.700333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.702365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.737220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/1986] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:04 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:04.738345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.749266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfbf2-89d9-425e-8c14-b709f4c2e3d6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.807009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.807682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.807966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.808242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.808971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.810905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.831887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1987] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:04 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:04.836177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.846850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331978f-3584-45bd-98a1-e3fabe1df9fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.851720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.852365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.852723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.853005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.853727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.855337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.865949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/1988] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:04.870199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.881346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8feb20-87c5-415e-9e2e-9ca43e3a3c10 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.886836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.887529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.887813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.888126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.888893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.890506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ca784-afd5-4f8f-87d2-42d4b502dabc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.901092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1989] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:04.906925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:04.917849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2faf27-656c-4231-a9b1-44816ad7fac9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:04.924022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:04.924937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.925300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.925566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] PATH_INFO: `/v3/projects/7cf72360d1bc42aab371730b33cf3ed4/users/96e07b9c15904d80b3f0f7425466a68b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:04.940608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf72360d1bc42aab371730b33cf3ed4, user_id=96e07b9c15904d80b3f0f7425466a68b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:04.943161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:04.962011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/1990] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:04 2022] PUT /identity/v3/projects/7cf72360d1bc42aab371730b33cf3ed4/users/96e07b9c15904d80b3f0f7425466a68b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:04.972702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:04.972986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:04.973264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.002083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] MFA Rules not processed for user `96e07b9c15904d80b3f0f7425466a68b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:05.003573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:05.023350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:05.073427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1991] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:05.077989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.139728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30302fdb-596d-4826-90ed-ab51ff536511 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.144827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.145478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.145764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.146045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.146747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.148714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.178617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/1992] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:05 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:05.182872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.194771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c86cca-1793-405f-82d8-b64c54d939f9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.257506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.258191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.258491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.258759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.259461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.261446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.284430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1993] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:05 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:05.289733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.305116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee89f3-402b-442b-aa58-c31e91ee184c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.311267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.311937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.312224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.312531 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.313272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.314997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.327617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/1994] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:05.332544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.342626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f167b-dd09-4e2a-94fa-7f1c5d667a6c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.347456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.348348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.348669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.348948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] PATH_INFO: `/v3/projects/551155d5f5ef401e93902d87f93ac8fe/users/f69aeba2e9dc43a99273a13080a8ae50/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.364282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551155d5f5ef401e93902d87f93ac8fe, user_id=f69aeba2e9dc43a99273a13080a8ae50, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.366990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.382289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1995] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:05 2022] PUT /identity/v3/projects/551155d5f5ef401e93902d87f93ac8fe/users/f69aeba2e9dc43a99273a13080a8ae50/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:05.386255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.393786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:05.397734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87534ed-dd8b-4636-ae46-f954472285e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.459412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.460087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.460374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.460684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.461412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.463109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90377175-0147-4335-9cee-11a7bc800d0d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.478935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/1996] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:05.480380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.495509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b95d1-dedd-447a-8db1-b9d590bda089 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.502318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.503851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.504189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.504727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] PATH_INFO: `/v3/projects/551155d5f5ef401e93902d87f93ac8fe/users/f69aeba2e9dc43a99273a13080a8ae50/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.506922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] RBAC: Authorizing `identity:create_grant(project_id=551155d5f5ef401e93902d87f93ac8fe, user_id=f69aeba2e9dc43a99273a13080a8ae50, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.509778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.524365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1997] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:05 2022] PUT /identity/v3/projects/551155d5f5ef401e93902d87f93ac8fe/users/f69aeba2e9dc43a99273a13080a8ae50/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:05.535742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.536083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.536368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.564245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] MFA Rules not processed for user `f69aeba2e9dc43a99273a13080a8ae50`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:05.565625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:05.643678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/1998] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:05.654414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:05.656179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:05.726488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd156-d245-4d65-a08a-187a0855275d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:05.730475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:05.731131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:05.731480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:05.731760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:05.759340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:05.790093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:05.791552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c014-7c6f-4099-808f-09f518e8330a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:05.801146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1999] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:07.004724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:07.006337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:07.088200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6cc8-2faf-42d4-8c24-059d3ef8f464 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:07.094599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:07.095925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:07.096252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:07.096712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] PATH_INFO: `/v3/users/6b929d7a2dc8473980dbe09faa484acd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:07.098225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b929d7a2dc8473980dbe09faa484acd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:07.101316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:07.135754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:07.157011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b929d7a2dc8473980dbe09faa484acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.157841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b929d7a2dc8473980dbe09faa484acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.158442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b929d7a2dc8473980dbe09faa484acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.179332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b929d7a2dc8473980dbe09faa484acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.190378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b929d7a2dc8473980dbe09faa484acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.207946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2000] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:07 2022] DELETE /identity/v3/users/6b929d7a2dc8473980dbe09faa484acd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:07.212698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:07.213590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:07.214043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:07.232494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:07.233724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:07.301693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/2001] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:07.314696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:07.316788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:07.400920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4489d-6dca-4139-9d89-ed33686214f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:07.404731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:07.405423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:07.405713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:07.405976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:07.415186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:07.416585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:07.426033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2002] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:07.717209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:07.718753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:07.731528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e177a3-13ee-483f-8d9a-5118c5f68240 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:07.736651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:07.738445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:07.738743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:07.739010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] PATH_INFO: `/v3/projects/9b4c8e691b724d86b21c6a30e5371dfb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:07.740263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4c8e691b724d86b21c6a30e5371dfb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:07.742270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:07.793598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4c8e691b724d86b21c6a30e5371dfb', 'request_id': 'req-42b48b15-40d8-428f-8de7-01a60561103f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.799817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4c8e691b724d86b21c6a30e5371dfb', 'request_id': 'req-42b48b15-40d8-428f-8de7-01a60561103f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.809308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4c8e691b724d86b21c6a30e5371dfb', 'request_id': 'req-42b48b15-40d8-428f-8de7-01a60561103f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.824862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] The token cache is being invalidate because project 9b4c8e691b724d86b21c6a30e5371dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:07.825156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.825823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.827420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/2003] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:07 2022] DELETE /identity/v3/projects/9b4c8e691b724d86b21c6a30e5371dfb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:07.831969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:07.842577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b992-64c6-4bd0-bb9f-1a41e93c2468 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:07.903377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:07.904276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:07.904611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:07.904898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] PATH_INFO: `/v3/users/a5d14b2bfc2b44f5bf0b204fe04016e5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:07.906210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d14b2bfc2b44f5bf0b204fe04016e5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:07.908270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:07.957824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d14b2bfc2b44f5bf0b204fe04016e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.958488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d14b2bfc2b44f5bf0b204fe04016e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.959076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d14b2bfc2b44f5bf0b204fe04016e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.961117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:07.974369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d14b2bfc2b44f5bf0b204fe04016e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:07.988137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d14b2bfc2b44f5bf0b204fe04016e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.001581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2004] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:07 2022] DELETE /identity/v3/users/a5d14b2bfc2b44f5bf0b204fe04016e5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:08.286988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.288703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.305183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48b15-40d8-428f-8de7-01a60561103f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.356303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:08.368529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.369426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.369953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.370442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] PATH_INFO: `/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.372034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83a51a1b3dd42a3b32021872ce77f9e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.374327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.429936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83a51a1b3dd42a3b32021872ce77f9e', 'request_id': 'req-172d5e7d-5299-4701-a750-263321774ec3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.430926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.432545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.436595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83a51a1b3dd42a3b32021872ce77f9e', 'request_id': 'req-172d5e7d-5299-4701-a750-263321774ec3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.443362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83a51a1b3dd42a3b32021872ce77f9e', 'request_id': 'req-172d5e7d-5299-4701-a750-263321774ec3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.445391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e44b50-e8c9-4fb5-9117-bbb072bb9ecf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.456883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] The token cache is being invalidate because project c83a51a1b3dd42a3b32021872ce77f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:08.457174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.457816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-172d5e7d-5299-4701-a750-263321774ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.459379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/2005] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:08 2022] DELETE /identity/v3/projects/c83a51a1b3dd42a3b32021872ce77f9e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:08.464063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ae92-598f-4dbd-9251-d07acf06d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.464349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ae92-598f-4dbd-9251-d07acf06d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.464684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ae92-598f-4dbd-9251-d07acf06d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.483217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad79ae92-598f-4dbd-9251-d07acf06d4c5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:08.484348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad79ae92-598f-4dbd-9251-d07acf06d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.503626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.504661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.504950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.505230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] PATH_INFO: `/v3/users/96e07b9c15904d80b3f0f7425466a68b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.506436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e07b9c15904d80b3f0f7425466a68b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.508366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.552403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/2006] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:08.554182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.554479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.554764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.556983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e07b9c15904d80b3f0f7425466a68b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.557786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e07b9c15904d80b3f0f7425466a68b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.558458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e07b9c15904d80b3f0f7425466a68b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.571071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e07b9c15904d80b3f0f7425466a68b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.573558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:08.574797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.579514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c57b1a1b-eeb8-412c-9eb3-808a0a44b8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e07b9c15904d80b3f0f7425466a68b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:08.582950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/2007] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:08.587070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.593671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2008] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:08 2022] DELETE /identity/v3/users/96e07b9c15904d80b3f0f7425466a68b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:08.595969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99b55a-e3a8-405b-9412-3f02f6c2a821 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.597936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.598229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.598505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.616530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:08.617736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.652223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.652920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.653204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.653479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.654207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:08.654513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.656122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.659048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:08.668796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/2009] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:10:08.673025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.679547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2010] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:08.683321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aec6-380e-453d-8216-646ef61c55a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.687805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.688807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.689090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.689181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.689366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.690137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.690676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:08.692433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.697458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:08.726406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/2011] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:08.727491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.749982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08419ca-f3df-4b3a-b0ee-9a7056cdbf93 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.790275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a57ab-8391-46f8-bd6f-96ac33f389e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.795706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.796430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.796706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.796827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.796971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.797457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.797682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.797786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.797984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.799721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.807869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.809019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.818644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2012] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:08.822019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/2013] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:08.826126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.835723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16993406-8797-42bb-85a6-ce6cec976ad7 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.840653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.841329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.841614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.841894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.842808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.844866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.857906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2014] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:08.860365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.871737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b09d9-c571-4159-b0ee-c12072c92be4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.878061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.878764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.879050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.879332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.880090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.881798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84cbcb-59f6-4f8d-acc3-ddee8d5a3ca0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.893073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/2015] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:08.897349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:08.907945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30df57-ac7e-4e5c-ae35-5ecc61bc0cf9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:08.913825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:08.914805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.915162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.915523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] PATH_INFO: `/v3/projects/edf9a0cf539e4df48b495e11a465b840/users/f1a824fd27c44e29a2866a5e26f44a79/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.932136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf9a0cf539e4df48b495e11a465b840, user_id=f1a824fd27c44e29a2866a5e26f44a79, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:08.934861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:08.948664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2016] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:08 2022] PUT /identity/v3/projects/edf9a0cf539e4df48b495e11a465b840/users/f1a824fd27c44e29a2866a5e26f44a79/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:08.959502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:08.959788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:08.960116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:08.988639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] MFA Rules not processed for user `f1a824fd27c44e29a2866a5e26f44a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:08.989874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:09.068109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/2017] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:09.079275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:09.081070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:09.144128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c69f4-bdfd-4258-843f-5e84ff5eac75 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:09.150627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:09.151630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:09.152001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:09.152216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:09.164545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:09.166288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:09.177045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2018] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:09.185995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:09.188136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:09.258330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:09.265446 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda9c08-f8f6-467c-ab1a-a66d3db802e9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:09.270578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:09.271459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:09.271742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:09.272015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] PATH_INFO: `/v3/projects/7cf72360d1bc42aab371730b33cf3ed4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:09.273247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf72360d1bc42aab371730b33cf3ed4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:09.275177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:09.325817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf72360d1bc42aab371730b33cf3ed4', 'request_id': 'req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.331592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf72360d1bc42aab371730b33cf3ed4', 'request_id': 'req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.338963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf72360d1bc42aab371730b33cf3ed4', 'request_id': 'req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.362348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] The token cache is being invalidate because project 7cf72360d1bc42aab371730b33cf3ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:09.362782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.363508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.365522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/2019] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:09 2022] DELETE /identity/v3/projects/7cf72360d1bc42aab371730b33cf3ed4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:09.369741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:09.382233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac1d5-cd34-49d0-9687-bf4745b272ae service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:09.392624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:09.472040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:09.473056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:09.473402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:09.473742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] PATH_INFO: `/v3/users/f69aeba2e9dc43a99273a13080a8ae50` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:09.475092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69aeba2e9dc43a99273a13080a8ae50)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:09.477220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:09.530514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69aeba2e9dc43a99273a13080a8ae50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.531185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69aeba2e9dc43a99273a13080a8ae50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.531778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69aeba2e9dc43a99273a13080a8ae50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.544770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69aeba2e9dc43a99273a13080a8ae50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.554025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc1f7a30-e7de-4c47-94aa-b843eef16c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69aeba2e9dc43a99273a13080a8ae50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.567043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2020] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:09 2022] DELETE /identity/v3/users/f69aeba2e9dc43a99273a13080a8ae50 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:09.841632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:09.843171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:09.859024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee125a8-a2fa-4909-841c-0c4a312ce6e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:09.918669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:09.919541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:09.919826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:09.920101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] PATH_INFO: `/v3/projects/551155d5f5ef401e93902d87f93ac8fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:09.921331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=551155d5f5ef401e93902d87f93ac8fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:09.923240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:09.974468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551155d5f5ef401e93902d87f93ac8fe', 'request_id': 'req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.980296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551155d5f5ef401e93902d87f93ac8fe', 'request_id': 'req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:09.986727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551155d5f5ef401e93902d87f93ac8fe', 'request_id': 'req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:10.001263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] The token cache is being invalidate because project 551155d5f5ef401e93902d87f93ac8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:10.001546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:10.002162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f1074c0-b1dc-4fb5-bc07-90afaad86f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:10.003881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/2021] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:09 2022] DELETE /identity/v3/projects/551155d5f5ef401e93902d87f93ac8fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:10.009102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.009393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.009664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.029277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:10.030512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:10.094537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2022] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:10.102404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.102683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.102965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.121940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:10.123527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:10.132636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/2023] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:10.137097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.146877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544dea1-17f0-4527-a0db-15082c90f8bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.151841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.152583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.152869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.153150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.153892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:10.154203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.155861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.168490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2024] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:10.172910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.182266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570f048-9d37-48de-b305-0d2381e1c3a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.187363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.188015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.188301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.188623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.189330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.191340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.195449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:10.220236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/2025] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:10 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:10.224676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.235069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753639c-10d6-474a-8ec2-1567e37edd15 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.253264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:10.301084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.301942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.302309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.302931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.303661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.305981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.332367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2026] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:10 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:10.333893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.344189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17215145-d835-43fe-94a3-4b7f5826ee85 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.349203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.349930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.350216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.350492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.351266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.353005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.366788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/2027] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:10.368190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.379060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae65b6-b986-4c0a-8f00-f41cc2f9cf35 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.383931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.384654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.384941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.385217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.385945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.387556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3e512-efda-4f6d-829a-aa521c4453fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.398371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2028] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:10.402476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.412466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a0b2-abf9-4aa5-92c4-136303df1082 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.417092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.417979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.418259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.418531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] PATH_INFO: `/v3/projects/1a43462c60384cecb441ccf14de5672e/users/41c4d3bab2164767b8eefcb4427b65b7/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.434013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a43462c60384cecb441ccf14de5672e, user_id=41c4d3bab2164767b8eefcb4427b65b7, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.436783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.451221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/2029] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:10 2022] PUT /identity/v3/projects/1a43462c60384cecb441ccf14de5672e/users/41c4d3bab2164767b8eefcb4427b65b7/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:10.456251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e79f-c276-4a36-b3ef-885c5beff169 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.456588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e79f-c276-4a36-b3ef-885c5beff169 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.456936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166e79f-c276-4a36-b3ef-885c5beff169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.474846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e166e79f-c276-4a36-b3ef-885c5beff169 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:10.476047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e166e79f-c276-4a36-b3ef-885c5beff169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:10.545148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2030] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:10.557775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:10.559465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:10.617842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc8d19-0498-4879-98f3-1f4ade418005 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:10.621906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:10.622909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:10.623196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:10.623470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:10.635327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:10.636949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:10.647363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/2031] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:15.027131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.027743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.027743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.056853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] MFA Rules not processed for user `41c4d3bab2164767b8eefcb4427b65b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:15.059016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:15.127017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2032] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:15.131654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.141828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4619-bcf2-438a-8bf7-9a648bf911ab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.146948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.147625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.147908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.148323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.149088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.151336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.184636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/2033] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:15 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:15.185935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.197337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9be3b5-2abd-4dc1-9547-94a28f7376bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.223473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:15.255827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.256548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.256835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.257112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.257817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.259769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.285565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2034] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:15 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:15.287201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.298084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea7276-4495-46f7-b627-7bbe2e984ed3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.303474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.304155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.304480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.304747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.305480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.307114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.319101 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/2035] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:15.323512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.334136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec1dc-f13c-41e9-a591-392f689b038e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.338875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.339799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.340076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.340375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] PATH_INFO: `/v3/projects/755877623fbf49d78f52dac475ac83a2/users/bed23ae7930b4be4b611160e42c343c4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.355879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] RBAC: Authorizing `identity:create_grant(project_id=755877623fbf49d78f52dac475ac83a2, user_id=bed23ae7930b4be4b611160e42c343c4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.358651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.373514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2036] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:15 2022] PUT /identity/v3/projects/755877623fbf49d78f52dac475ac83a2/users/bed23ae7930b4be4b611160e42c343c4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:15.377646 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.387945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6ed2-fbe9-4a56-9b39-1ed6500f9ea0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.449988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.450671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.450949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.451237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.451969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.453670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482daea-532d-42cf-ada5-7b8d7bd4ea39 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.464667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/2037] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:15.469163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:15.485196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0972a-475e-49f6-8c5c-76ad5f24635f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:15.491145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:15.492075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:15.492367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:15.492692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] PATH_INFO: `/v3/projects/755877623fbf49d78f52dac475ac83a2/users/bed23ae7930b4be4b611160e42c343c4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:15.494732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755877623fbf49d78f52dac475ac83a2, user_id=bed23ae7930b4be4b611160e42c343c4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:15.497776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:15.509542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2038] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:15 2022] PUT /identity/v3/projects/755877623fbf49d78f52dac475ac83a2/users/bed23ae7930b4be4b611160e42c343c4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:20.894630 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:20.895549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:20.895823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:20.946685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] MFA Rules not processed for user `bed23ae7930b4be4b611160e42c343c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:20.947919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:20.953719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:21.032066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/2039] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:21.046254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:21.047978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:21.140567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9bbe9-a833-4c14-9ccb-659d9fab4745 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:21.152618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:21.153302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:21.153592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:21.153872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:21.214596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:21.216245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010794-7c46-4b5d-aaf9-21e6270f44e5 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:21.226795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2040] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:27.914723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:27.916547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:27.994935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74207e7-9ce1-4fd4-86b4-0e86058ba984 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:28.002938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:28.004204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.004628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.004946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] PATH_INFO: `/v3/users/61cae71e698f4e08863f6deac5ea1351` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.006632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61cae71e698f4e08863f6deac5ea1351)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:28.008978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:28.083990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61cae71e698f4e08863f6deac5ea1351'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.085236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61cae71e698f4e08863f6deac5ea1351'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.108471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61cae71e698f4e08863f6deac5ea1351'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.109303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61cae71e698f4e08863f6deac5ea1351'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.117836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61cae71e698f4e08863f6deac5ea1351'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.138757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/2041] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:27 2022] DELETE /identity/v3/users/61cae71e698f4e08863f6deac5ea1351 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:28.143057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.143652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.143762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.170706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:28.171857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:28.194782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:28.249056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2042] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:28.259835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:28.261338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:28.287606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:28.320661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db8664-2134-4aa6-986a-5bfedc83ce39 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:28.324620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:28.325276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.325563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.325835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.335223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:28.336710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64ba84-1b82-457e-92a4-1733e6432fcc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:28.346899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/2043] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:28.667814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:28.669498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:28.690848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7abcc-e667-4e97-8b03-c396a60329fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:28.696716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:28.697857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.698252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.698623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] PATH_INFO: `/v3/projects/740d1155ea774f03880d2b3ce2278874` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.700097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] RBAC: Authorizing `identity:delete_project(project_id=740d1155ea774f03880d2b3ce2278874)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:28.702050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:28.752285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740d1155ea774f03880d2b3ce2278874', 'request_id': 'req-3ec2603c-e439-48c4-a048-4b707277792d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.757943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740d1155ea774f03880d2b3ce2278874', 'request_id': 'req-3ec2603c-e439-48c4-a048-4b707277792d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.766879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740d1155ea774f03880d2b3ce2278874', 'request_id': 'req-3ec2603c-e439-48c4-a048-4b707277792d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.779371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] The token cache is being invalidate because project 740d1155ea774f03880d2b3ce2278874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:28.779664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.780402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3ec2603c-e439-48c4-a048-4b707277792d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:28.782071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2044] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:28 2022] DELETE /identity/v3/projects/740d1155ea774f03880d2b3ce2278874 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:28.787095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.787372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.787652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.805594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:28.806731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:28.873206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/2045] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:28.881630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.882096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.882497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.901848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:28.903296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:28.915887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2046] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:28.917735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:28.928201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15e4b8-fcf4-4c05-b8d1-c50ca9aeb327 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:28.934525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:28.935196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.935475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.935749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.936519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:28.936814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:28.938558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:28.954116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/2047] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:28.958779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:28.968272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443df0d1-6e51-47d7-8d90-a925b0ed5760 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:28.974727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:28.975462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:28.975833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:28.976196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:28.976950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:28.978899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.008650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2048] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:28 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:29.013532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.023730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09896b9e-94f8-497a-bdd0-e59851c195d5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.086704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.087408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.087672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.087974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.088664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.091501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.121794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2049] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:29 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:29.126343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.138760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e7fe4-c9cb-4ea2-a49e-b99653aefaee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.150600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.151598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.152041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.152386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.154326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.156246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.171644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2050] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:29.173235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.190814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a7f8c-b315-47e5-ba2b-b61d5d068205 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.196549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.197461 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.197934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.198227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.199139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.201147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb80b2-f757-47a4-a2c4-f4f5b20dbbb6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.221992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2051] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:29.226067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.243228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bd900-77ef-463f-8d4d-46155cba992a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.256936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.258177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.258466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.258744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] PATH_INFO: `/v3/projects/bbee283972514456add5099819c9e86a/users/3f10016238934d9d8d959f5fb88cf321/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.295668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbee283972514456add5099819c9e86a, user_id=3f10016238934d9d8d959f5fb88cf321, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.299015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.324707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2052] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:29 2022] PUT /identity/v3/projects/bbee283972514456add5099819c9e86a/users/3f10016238934d9d8d959f5fb88cf321/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:29.342529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.342803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.343160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.382002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] MFA Rules not processed for user `3f10016238934d9d8d959f5fb88cf321`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:29.383959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:29.406768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:29.482889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2053] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:29.487497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.530543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:29.556176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401b16-8372-4d65-a2d8-19b08e984806 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.561252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.561955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.562291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.562557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.563270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.565753 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.604075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2054] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:29 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:29.605196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.619936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d89ad6-70f8-4b05-9969-d160d6b7f351 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.699919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.699919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.699919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.700535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.700866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.702801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.734957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2055] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:29 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:29.739548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.761663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c63be-4354-4b1d-95cf-db0ae6c6bc54 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.767020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.767668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.767960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.768503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.769324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.771638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.790723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2056] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:29.796085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.807121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907149a-7e6b-4f9d-8bf9-4dfb28f846e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.812330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a46797-48cc-4506-a613-69208176faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.813058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46797-48cc-4506-a613-69208176faef admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.813341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46797-48cc-4506-a613-69208176faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.813874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a46797-48cc-4506-a613-69208176faef admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.814643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a46797-48cc-4506-a613-69208176faef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.816350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a46797-48cc-4506-a613-69208176faef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.827903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2057] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:29.833163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:29.845138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea767c-c6e6-4120-89d5-373ea74d7837 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:29.850998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:29.851922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.852208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.852535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] PATH_INFO: `/v3/projects/96145340203442378fb4e3febaef82db/users/1989fef8e41e44e09cef20931a6b3a50/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.870650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96145340203442378fb4e3febaef82db, user_id=1989fef8e41e44e09cef20931a6b3a50, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:29.873767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:29.890278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2058] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:29 2022] PUT /identity/v3/projects/96145340203442378fb4e3febaef82db/users/1989fef8e41e44e09cef20931a6b3a50/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:29.900687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:29.900976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:29.901252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:29.939201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] MFA Rules not processed for user `1989fef8e41e44e09cef20931a6b3a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:29.942448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:30.027523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2059] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:30.033001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.105593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bff980-f8a1-4444-8141-172c1e3a7293 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.113988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.115115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.116584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.117322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.119657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.124151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.160624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2060] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:30 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:30.166423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.180150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfab163-b9f4-4f6a-b656-8703c0bba37a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.217693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:30.255531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.256217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.256514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.256814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.257540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.259477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.284132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2061] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:30 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:30.288304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.299954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9755f-ecce-4474-a8df-2ef08a10f6c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.308556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.309623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.309937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.310216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.311624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.313973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.329422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2062] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:30.334119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.346065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f290a-a364-424f-a80f-3f4ae2c80087 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.351323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.352231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.352525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.352831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] PATH_INFO: `/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97/users/a854f08ff3f24da48b0e43784270cd71/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.370428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca4b7a74c924e6b80e3bd08e8bdfc97, user_id=a854f08ff3f24da48b0e43784270cd71, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.373510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.388800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2063] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:30 2022] PUT /identity/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97/users/a854f08ff3f24da48b0e43784270cd71/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:30.394340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.405838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd25262-212e-4ed3-a49f-1b2480531734 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.472732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.473532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.473821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.474336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.475078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.476885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b881-e32a-46d3-9a74-ffc08c34b3cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.482921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:30.492850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2064] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:30.497837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.510832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f5dac-7625-405f-8301-58415eddbe02 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.516604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.517522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.517806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.518250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] PATH_INFO: `/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97/users/a854f08ff3f24da48b0e43784270cd71/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.523540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca4b7a74c924e6b80e3bd08e8bdfc97, user_id=a854f08ff3f24da48b0e43784270cd71, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.526471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.544510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2065] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:30 2022] PUT /identity/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97/users/a854f08ff3f24da48b0e43784270cd71/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:30.552067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.552348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.552661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.580602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] MFA Rules not processed for user `a854f08ff3f24da48b0e43784270cd71`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:30.581818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:30.659457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2066] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:30.671014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:30.673030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:30.733083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413e86-22d0-456c-ad39-f53c13f65e99 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:30.737165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:30.737793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:30.738074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:30.738347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:30.797411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:30.798934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:30.808169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2067] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:33.380663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:33.391084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdefb0-089f-4879-967c-70f45f10242a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:33.394866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:33.395588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:33.395943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:33.396292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:33.459571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:33.461233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:33.472258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2068] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:35.531784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:35.533366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:35.575364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:35.601270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e566ed-a90e-40f1-bbe5-18606e0e6f5e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:35.605968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:35.606843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:35.607154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:35.607427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] PATH_INFO: `/v3/users/e3ae9f551626460e864561c613d5f273` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:35.608722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ae9f551626460e864561c613d5f273)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:35.610690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:35.664006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ae9f551626460e864561c613d5f273'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:35.664763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ae9f551626460e864561c613d5f273'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:35.676776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ae9f551626460e864561c613d5f273'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:35.677507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ae9f551626460e864561c613d5f273'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:35.684917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02f8a8ba-ac33-4f15-8e88-4beacb839bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ae9f551626460e864561c613d5f273'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:35.699503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2069] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:35 2022] DELETE /identity/v3/users/e3ae9f551626460e864561c613d5f273 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:35.881006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:35.892333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2e2ec-b727-405c-b7a0-94a87b249d3e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:35.949038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:35.949866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:35.950154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:35.950433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] PATH_INFO: `/v3/projects/8c102f5ee47e426c8f4e0ced15f60bee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:35.951711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c102f5ee47e426c8f4e0ced15f60bee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:35.953777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:35.978285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:36.003599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c102f5ee47e426c8f4e0ced15f60bee', 'request_id': 'req-d8a16522-ff8c-4e2e-a774-24be3dca9f03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:36.009299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c102f5ee47e426c8f4e0ced15f60bee', 'request_id': 'req-d8a16522-ff8c-4e2e-a774-24be3dca9f03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:36.018178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c102f5ee47e426c8f4e0ced15f60bee', 'request_id': 'req-d8a16522-ff8c-4e2e-a774-24be3dca9f03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:36.029554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] The token cache is being invalidate because project 8c102f5ee47e426c8f4e0ced15f60bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:36.029848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:36.030488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8a16522-ff8c-4e2e-a774-24be3dca9f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:36.035445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2070] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:35 2022] DELETE /identity/v3/projects/8c102f5ee47e426c8f4e0ced15f60bee => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:36.037885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.038173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.038439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.058043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:36.059434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:36.121687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2071] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:36.130063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.130356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.130635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.153823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:36.155211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:36.164792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2072] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:36.169167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.179064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f49a3-b804-45c6-a6ce-ac787cc5d240 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.183659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.184308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.184631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.184909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.185674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:36.185984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.187598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.199187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2073] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:36.203410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.213295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a69d49-595c-4170-88ce-08bc7ea87348 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.219142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.219801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.220084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.220361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.221179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.223098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.251786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2074] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:36 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:36.256208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.266191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4de35-b99c-47fc-8963-d74448288ef4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.312054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:36.340691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.341733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.342172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.342624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.343611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.346637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.368591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2075] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:36 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:36.373415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.384565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515eb7b-b9d0-4102-ac5f-8e39c8ccdc9e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.389750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.390442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.390720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.390994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.391720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.393431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.405433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2076] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:36.410332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.424331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079717d-99c9-4969-a537-8957e8c6cab6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.430539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.431469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.431469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.431710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.432566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.434247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60f49-9c23-43be-bc39-cc1d81af85ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.445365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2077] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:36.449734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.460634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d0c46-78f5-4351-be1e-33ca0bea598c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.465585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.466487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.466773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.467059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] PATH_INFO: `/v3/projects/e986ed3e936e47bb8c4ac70fb24bb8ef/users/81fa4d4cc93340c1a1d38828e027a5b5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.482228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e986ed3e936e47bb8c4ac70fb24bb8ef, user_id=81fa4d4cc93340c1a1d38828e027a5b5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.485013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.502838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2078] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:36 2022] PUT /identity/v3/projects/e986ed3e936e47bb8c4ac70fb24bb8ef/users/81fa4d4cc93340c1a1d38828e027a5b5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:36.513959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.514278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.514671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.545046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] MFA Rules not processed for user `81fa4d4cc93340c1a1d38828e027a5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:36.546264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:36.618612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2079] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:36.630172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.631873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:36.723725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b2b52-d42b-4625-b7d8-6c1fd456ac98 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.729674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.730327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.730671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.730927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.739262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:36.741059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:36.755879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2080] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:36.899140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:36.900775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:36.967418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a126a96-159f-4356-b41c-511554fb4b63 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:36.972023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:36.972710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:36.972998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:36.973275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:36.975111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:37.048727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:37.050490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca807250-cc29-40bf-a009-6d8bccffaffc service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:37.056294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:37.063525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2081] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:41.656578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:41.699797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:41.730053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbee1a-2cb8-421f-9080-27761963aaa7 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:41.734776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:41.735660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:41.735947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:41.736296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] PATH_INFO: `/v3/users/81fa4d4cc93340c1a1d38828e027a5b5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:41.737639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81fa4d4cc93340c1a1d38828e027a5b5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:41.739567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:41.791150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81fa4d4cc93340c1a1d38828e027a5b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:41.791872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81fa4d4cc93340c1a1d38828e027a5b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:41.792580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81fa4d4cc93340c1a1d38828e027a5b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:41.805360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81fa4d4cc93340c1a1d38828e027a5b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:41.813660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81fa4d4cc93340c1a1d38828e027a5b5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:41.826543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2082] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:41 2022] DELETE /identity/v3/users/81fa4d4cc93340c1a1d38828e027a5b5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:41.831340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:41.831816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:41.831933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:41.848965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:41.850179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:41.912923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2083] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:41.922624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:41.924363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:41.985520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fbaf5-e601-4a64-af3a-1c7a5c8fd30e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:41.989469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:41.990123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:41.990401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:41.990680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.000503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.001975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66acf4b-85a7-4499-a2da-7ff4d2b0fe8b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.012654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2084] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:42.314131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.315728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:42.327964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca85626-2730-4924-986e-f9dfc6d44cb3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.333425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.334586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.334919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.335371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] PATH_INFO: `/v3/projects/e986ed3e936e47bb8c4ac70fb24bb8ef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.336816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e986ed3e936e47bb8c4ac70fb24bb8ef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.339187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.399655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e986ed3e936e47bb8c4ac70fb24bb8ef', 'request_id': 'req-fb995607-3d13-4239-9bac-37d63ee96ad5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:42.405282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e986ed3e936e47bb8c4ac70fb24bb8ef', 'request_id': 'req-fb995607-3d13-4239-9bac-37d63ee96ad5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:42.411785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e986ed3e936e47bb8c4ac70fb24bb8ef', 'request_id': 'req-fb995607-3d13-4239-9bac-37d63ee96ad5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:42.428108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] The token cache is being invalidate because project e986ed3e936e47bb8c4ac70fb24bb8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:42.428343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:42.429014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb995607-3d13-4239-9bac-37d63ee96ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:42.430594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2085] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:42 2022] DELETE /identity/v3/projects/e986ed3e936e47bb8c4ac70fb24bb8ef => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:42.435604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.435895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.436172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.453883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:42.455704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:42.505999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:42.526126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2086] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:42.529938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.530235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.530518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.550291 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:42.551539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:42.560130 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2087] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:42.564674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.576113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1186c4-e2e8-45b5-b91f-84cdf5b1d243 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.582634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.583352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.583647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.583930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.584734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:42.585052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.586723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.599470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2088] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:42.604532 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.614932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fe2c0-c7e2-4ca8-a21b-b82175da2222 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.621423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.622369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.622676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.622946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.623775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.626075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.657957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2089] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:42 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:42.662623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.673852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e271852-10d0-492b-a385-674493d4d66c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.744106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.744924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.745212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.745489 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.746248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.748705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.773031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2090] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:42 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:42.777661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.789388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1859-69ca-49f4-845e-2b01103f71c4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.794619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.795454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.795768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.796119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.796942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.798669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.816617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2091] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:42.819301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.830912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7f99-d2a8-41b1-8c7a-15ed1a881d9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.838498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.839192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.839502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.839773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.840578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.842401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee02d15-885b-4553-a945-45cf42f5de08 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.865156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2092] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:42.870277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:42.881676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f773a99-ad61-479b-8bb4-0c789580d609 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:42.887043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:42.888116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.888492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.888833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] PATH_INFO: `/v3/projects/9de162af8fab4bc488ca05f5e3003d9b/users/bef96ebe28eb47c89121c0e23a9f538e/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.904908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de162af8fab4bc488ca05f5e3003d9b, user_id=bef96ebe28eb47c89121c0e23a9f538e, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:42.907796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:42.923254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2093] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:42 2022] PUT /identity/v3/projects/9de162af8fab4bc488ca05f5e3003d9b/users/bef96ebe28eb47c89121c0e23a9f538e/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:42.934985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:42.935283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:42.935566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:42.965208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] MFA Rules not processed for user `bef96ebe28eb47c89121c0e23a9f538e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:42.966478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:43.050655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2094] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:43.055592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.079429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:43.131725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b860-3b4d-4d5c-a7a6-261edd8aaa76 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.136631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.137309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.137594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.137869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.138562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.140530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.169055 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2095] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:43 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:43.174267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.185172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92dcd2-e04e-4aa3-93b7-818210830716 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.234440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:43.246714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.247924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.248210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.248549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.249819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.252064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.280302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2096] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:43 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:43.281661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.294357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718c209-09a1-4b59-8b7d-7232f9a8acba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.300923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.301611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.301901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.302181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.302924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.304624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.316106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2097] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:43.320677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.333527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe937f-5fb5-430e-a247-84c73da179fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.339092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.340000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.340281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.340594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] PATH_INFO: `/v3/projects/92340f1b9d7943748d01cb7ef3128da7/users/92829bc33c3245ce95231ea7ac73abcc/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.356953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=92340f1b9d7943748d01cb7ef3128da7, user_id=92829bc33c3245ce95231ea7ac73abcc, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.359930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.373809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2098] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:43 2022] PUT /identity/v3/projects/92340f1b9d7943748d01cb7ef3128da7/users/92829bc33c3245ce95231ea7ac73abcc/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:43.378097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.396726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00156420-795d-4238-bf60-93543f5d34ff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.476841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.477578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.477853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.478190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.479011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.480911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df2e17-b2a3-4232-897c-55c9125719dd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.492527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2099] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:10:43.496640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.510634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5d330-123c-4e1e-ab40-dda844074eee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.515606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.516567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.516885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.517195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] PATH_INFO: `/v3/projects/92340f1b9d7943748d01cb7ef3128da7/users/92829bc33c3245ce95231ea7ac73abcc/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.519393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92340f1b9d7943748d01cb7ef3128da7, user_id=92829bc33c3245ce95231ea7ac73abcc, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.522264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.543858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2100] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:43 2022] PUT /identity/v3/projects/92340f1b9d7943748d01cb7ef3128da7/users/92829bc33c3245ce95231ea7ac73abcc/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:43.545863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.546194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.546484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.581375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] MFA Rules not processed for user `92829bc33c3245ce95231ea7ac73abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:43.582919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:43.658527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2101] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:43.669836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:43.671741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:43.733081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262442b-4fa3-4934-9d72-40f1585d7b94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:43.737076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:43.737681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:43.737965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:43.738241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:43.748425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:43.750042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:43.760970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2102] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:44.251723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:44.253511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.265365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:44.290325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:44.346680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52c618-d382-4e0c-9bd2-2831997e16cf service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:44.351824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:44.352766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.353110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.353397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] PATH_INFO: `/v3/users/bef96ebe28eb47c89121c0e23a9f538e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.354641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef96ebe28eb47c89121c0e23a9f538e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:44.356690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:44.360021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1b8c-e806-4b1a-b7ea-48f8b6e512a7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:44.364826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:44.365887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.366215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.366605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] PATH_INFO: `/v3/users/3f10016238934d9d8d959f5fb88cf321` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.367824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f10016238934d9d8d959f5fb88cf321)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:44.369997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:44.408892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef96ebe28eb47c89121c0e23a9f538e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.409660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef96ebe28eb47c89121c0e23a9f538e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.410267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef96ebe28eb47c89121c0e23a9f538e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.411451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:44.420142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f10016238934d9d8d959f5fb88cf321'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.420843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f10016238934d9d8d959f5fb88cf321'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.425903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef96ebe28eb47c89121c0e23a9f538e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.433739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f10016238934d9d8d959f5fb88cf321'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.434427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f10016238934d9d8d959f5fb88cf321'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.434547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9856badf-4dba-4c97-96f6-4082f247dbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef96ebe28eb47c89121c0e23a9f538e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.441458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3763edf-f4dd-4e37-a9e0-01becabc0fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f10016238934d9d8d959f5fb88cf321'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.447921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2103] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:44 2022] DELETE /identity/v3/users/bef96ebe28eb47c89121c0e23a9f538e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:44.452233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.452556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.452833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.455228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2104] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:44 2022] DELETE /identity/v3/users/3f10016238934d9d8d959f5fb88cf321 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:44.459404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.459760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.460099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.469600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:44.470899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.477628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:44.478826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.531074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2105] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:44.540540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:44.541063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2106] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:44.541984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.549584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:44.550901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.591339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:44.603019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0c2aa-2168-4d50-b027-bd31ff95fa54 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:44.606931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:44.607579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.607862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.608136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.613926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575a13-29d5-4fad-8c90-e5020f2f008d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:44.617605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:44.618264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.618544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.618762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:44.618884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.620514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:44.630462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:44.632010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06971c11-b37d-4b0b-a90b-dc47c7d82d25 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:44.637512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2107] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:44.647915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2108] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:44.762730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:44.764461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:44.780762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f866a47-8309-4cc5-9dcb-d002058edc2a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:44.786608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:44.787514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.787808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.788087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] PATH_INFO: `/v3/users/f1a824fd27c44e29a2866a5e26f44a79` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.789449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a824fd27c44e29a2866a5e26f44a79)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:44.791433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:44.852709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a824fd27c44e29a2866a5e26f44a79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.853377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a824fd27c44e29a2866a5e26f44a79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.853985 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a824fd27c44e29a2866a5e26f44a79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.869666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:44.871672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a824fd27c44e29a2866a5e26f44a79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.885811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a824fd27c44e29a2866a5e26f44a79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:44.915143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2109] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:44 2022] DELETE /identity/v3/users/f1a824fd27c44e29a2866a5e26f44a79 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:44.919657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:44.919959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:44.920263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:44.956007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:44.957442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.032276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2110] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:45.042869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.044794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.089097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.092644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.111361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521443d-ece1-4e12-80ab-9f61aa505b26 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.113959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9b087-fbda-4351-8868-902b22ee3717 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.118334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.119067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.119353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.119642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.125177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.126037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.126389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.126847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] PATH_INFO: `/v3/projects/9de162af8fab4bc488ca05f5e3003d9b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.128909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de162af8fab4bc488ca05f5e3003d9b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.132790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.134198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.135851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.146769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2111] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:45.147746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.149330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.162875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff691989-8429-4d9b-9228-23045b5fad73 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.168599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.169728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.170334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.171353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] PATH_INFO: `/v3/projects/bbee283972514456add5099819c9e86a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.173032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbee283972514456add5099819c9e86a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.175072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.229770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de162af8fab4bc488ca05f5e3003d9b', 'request_id': 'req-0606323b-47a0-48f1-b271-47109ebe8577'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.234576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbee283972514456add5099819c9e86a', 'request_id': 'req-e962b364-89e2-4718-89d9-357088661801'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.238735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de162af8fab4bc488ca05f5e3003d9b', 'request_id': 'req-0606323b-47a0-48f1-b271-47109ebe8577'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.246902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbee283972514456add5099819c9e86a', 'request_id': 'req-e962b364-89e2-4718-89d9-357088661801'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.257650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de162af8fab4bc488ca05f5e3003d9b', 'request_id': 'req-0606323b-47a0-48f1-b271-47109ebe8577'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.258256 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbee283972514456add5099819c9e86a', 'request_id': 'req-e962b364-89e2-4718-89d9-357088661801'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.273516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] The token cache is being invalidate because project bbee283972514456add5099819c9e86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:45.273819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.274506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.277755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2112] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:45 2022] DELETE /identity/v3/projects/bbee283972514456add5099819c9e86a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:45.279417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] The token cache is being invalidate because project 9de162af8fab4bc488ca05f5e3003d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:45.279718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.281170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.281790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.283495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.284198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2113] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:45 2022] DELETE /identity/v3/projects/9de162af8fab4bc488ca05f5e3003d9b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:45.287992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.290948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.370305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962b364-89e2-4718-89d9-357088661801 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.380981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.383340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.383631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.384055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] PATH_INFO: `/v3/users/1989fef8e41e44e09cef20931a6b3a50` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.385634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1989fef8e41e44e09cef20931a6b3a50)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.387968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.394772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:45.405638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606323b-47a0-48f1-b271-47109ebe8577 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.413138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.414247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.414844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.414956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] PATH_INFO: `/v3/users/92829bc33c3245ce95231ea7ac73abcc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.416304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=92829bc33c3245ce95231ea7ac73abcc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.419735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.423508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:45.485727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1989fef8e41e44e09cef20931a6b3a50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.487290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1989fef8e41e44e09cef20931a6b3a50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.488045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1989fef8e41e44e09cef20931a6b3a50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.505743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1989fef8e41e44e09cef20931a6b3a50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.518487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92829bc33c3245ce95231ea7ac73abcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.519411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92829bc33c3245ce95231ea7ac73abcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.521511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1989fef8e41e44e09cef20931a6b3a50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.538230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92829bc33c3245ce95231ea7ac73abcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.539240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92829bc33c3245ce95231ea7ac73abcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.550348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-34168509-c510-4947-913a-dfd46dc5e5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92829bc33c3245ce95231ea7ac73abcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.550794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2114] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:45 2022] DELETE /identity/v3/users/1989fef8e41e44e09cef20931a6b3a50 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:45.553854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:45.562179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:45.600516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2115] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:45 2022] DELETE /identity/v3/users/92829bc33c3245ce95231ea7ac73abcc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:45.662820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c197213-e73a-4e84-afc5-7028b280e378 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:45.766474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:45.768298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.769311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.769685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] PATH_INFO: `/v3/projects/edf9a0cf539e4df48b495e11a465b840` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.772577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf9a0cf539e4df48b495e11a465b840)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:45.774884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:45.840631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:45.857742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf9a0cf539e4df48b495e11a465b840', 'request_id': 'req-131152e6-db2b-484c-a9e0-f7b463e8b240'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.865483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf9a0cf539e4df48b495e11a465b840', 'request_id': 'req-131152e6-db2b-484c-a9e0-f7b463e8b240'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.876470 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf9a0cf539e4df48b495e11a465b840', 'request_id': 'req-131152e6-db2b-484c-a9e0-f7b463e8b240'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.891013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] The token cache is being invalidate because project edf9a0cf539e4df48b495e11a465b840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:45.891928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.893307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-131152e6-db2b-484c-a9e0-f7b463e8b240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:45.894891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2116] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:45 2022] DELETE /identity/v3/projects/edf9a0cf539e4df48b495e11a465b840 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:45.900699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:45.901132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:45.901548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:45.924747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:45.926016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.025048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2117] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:46.036059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.036751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.037378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.043106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.046069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.060376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2f970-bc42-4bdc-9215-bc8b87eaf747 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.072373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:46.073797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.083808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2118] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:46.086853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.087678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.088785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.089958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.090424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] PATH_INFO: `/v3/projects/96145340203442378fb4e3febaef82db` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.091948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=96145340203442378fb4e3febaef82db)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.094520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.113746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd58d6f-33a4-4669-9b6b-e70a1c29ea63 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.119281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.120317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.120625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.120907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.121829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:46.122427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.124384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.142651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2119] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:10:46.143570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.154453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82bcae-edef-4520-83c1-2e4e071605a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.163051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96145340203442378fb4e3febaef82db', 'request_id': 'req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.177204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96145340203442378fb4e3febaef82db', 'request_id': 'req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.187503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96145340203442378fb4e3febaef82db', 'request_id': 'req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.203986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] The token cache is being invalidate because project 96145340203442378fb4e3febaef82db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:46.204438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.206387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.208493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2120] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:46 2022] DELETE /identity/v3/projects/96145340203442378fb4e3febaef82db => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:46.211441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.212335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.218984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:46.267163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.268085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.268381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.268695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.269413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.271503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.303649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2121] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:46.303649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.303978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c4bbd-87ba-4fc2-8b14-1352761224ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.304565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.367179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.368060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.368345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.368653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] PATH_INFO: `/v3/projects/92340f1b9d7943748d01cb7ef3128da7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.369916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92340f1b9d7943748d01cb7ef3128da7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.371844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.373381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ff88-e21f-400c-ab67-873d7efcc995 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.380986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.381929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.382215 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.382492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] PATH_INFO: `/v3/users/a854f08ff3f24da48b0e43784270cd71` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.383722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a854f08ff3f24da48b0e43784270cd71)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.385893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.423294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92340f1b9d7943748d01cb7ef3128da7', 'request_id': 'req-e55ba79b-41a7-45b9-81cc-12552b6a604b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.426249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:46.429569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92340f1b9d7943748d01cb7ef3128da7', 'request_id': 'req-e55ba79b-41a7-45b9-81cc-12552b6a604b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.437374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92340f1b9d7943748d01cb7ef3128da7', 'request_id': 'req-e55ba79b-41a7-45b9-81cc-12552b6a604b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.442602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a854f08ff3f24da48b0e43784270cd71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.443441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a854f08ff3f24da48b0e43784270cd71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.444207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a854f08ff3f24da48b0e43784270cd71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.451458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] The token cache is being invalidate because project 92340f1b9d7943748d01cb7ef3128da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:46.451948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.452638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.454222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2122] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:46 2022] DELETE /identity/v3/projects/92340f1b9d7943748d01cb7ef3128da7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:46.455340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.457762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.458818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a854f08ff3f24da48b0e43784270cd71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.467327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1eba2676-527c-4917-9a91-55ff974a9cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a854f08ff3f24da48b0e43784270cd71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:46.482595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2123] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:46 2022] DELETE /identity/v3/users/a854f08ff3f24da48b0e43784270cd71 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:46.485639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be9f9-6941-46dd-b8cb-bca65e4f4105 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.485639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be9f9-6941-46dd-b8cb-bca65e4f4105 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.485639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be9f9-6941-46dd-b8cb-bca65e4f4105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.505498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-225be9f9-6941-46dd-b8cb-bca65e4f4105 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:46.506860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-225be9f9-6941-46dd-b8cb-bca65e4f4105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.507345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:46.532716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ba79b-41a7-45b9-81cc-12552b6a604b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.581352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2124] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:46.589379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.589752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.590102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.598613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.599298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.599582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.599905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.600582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.602523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.609615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:46.610852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.620104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2125] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:46.624353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.634204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb1254-9572-461f-a155-93793cfd0807 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.636742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2126] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:46.640910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.642431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.643109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.643397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.643680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.644469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:46.644820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.646540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.661646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2127] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:46.664846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.671349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37bd6-92c6-4009-bad0-5204832b58fb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.676557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527283-606c-4051-ae46-33717ff1ab20 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.678586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.679453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.679741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.680095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.681014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.682516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.682936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.683557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.683989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.684495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.685369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.687617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.698180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2128] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:46.699280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.710393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477c7b-eb64-478c-a9ae-5ccc496c62e4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.715659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.717104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.717398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.717685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.718504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.720118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.721406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2129] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:46.725007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.736469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2130] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:46.739378 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.750828 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b9286-8b9d-407b-9b77-3ae86ad50e0c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.806542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b145ac-78fe-41d6-b4b0-7a0f03c0d54c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.815177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.815981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.816335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.816733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.817645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.819661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.820368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.821565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.822025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.822475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] PATH_INFO: `/v3/projects/0d8e2178a3c84520ac00fc1cf2da0329/users/4d0e5c4a49d84ab9ab4c2db7d09e6fac/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.831943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:46.841768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8e2178a3c84520ac00fc1cf2da0329, user_id=4d0e5c4a49d84ab9ab4c2db7d09e6fac, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.844585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.850702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2131] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:46.851690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.853501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.867289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2132] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:46 2022] PUT /identity/v3/projects/0d8e2178a3c84520ac00fc1cf2da0329/users/4d0e5c4a49d84ab9ab4c2db7d09e6fac/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:46.867289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.871403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7984ac-1f5f-4ded-8bf6-7e1303a58780 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.910571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:46.932165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380e669-9053-4f33-b0a2-d0e540881bd2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:46.937264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.937942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.938163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:46.938280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.938488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.939044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.939296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.939856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.940143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] PATH_INFO: `/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.940966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c793a89-e2da-4fe4-b334-ca10019eeb9d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.941823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca4b7a74c924e6b80e3bd08e8bdfc97)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:46.943270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:46.951275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2133] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:46.955623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494577-82e4-4c91-b1db-04107c039394 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:46.956439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494577-82e4-4c91-b1db-04107c039394 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:46.956439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97494577-82e4-4c91-b1db-04107c039394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:46.973572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-97494577-82e4-4c91-b1db-04107c039394 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:46.974817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97494577-82e4-4c91-b1db-04107c039394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:46.985549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2134] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:46.985549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494577-82e4-4c91-b1db-04107c039394 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:46.993485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97494577-82e4-4c91-b1db-04107c039394 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.004494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca4b7a74c924e6b80e3bd08e8bdfc97', 'request_id': 'req-e7c1ab8d-3376-4f54-9cda-25e53e309daf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:47.010873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca4b7a74c924e6b80e3bd08e8bdfc97', 'request_id': 'req-e7c1ab8d-3376-4f54-9cda-25e53e309daf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:47.019663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca4b7a74c924e6b80e3bd08e8bdfc97', 'request_id': 'req-e7c1ab8d-3376-4f54-9cda-25e53e309daf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:47.031741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] The token cache is being invalidate because project 5ca4b7a74c924e6b80e3bd08e8bdfc97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:47.032025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:47.032708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:47.034321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2135] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:46 2022] DELETE /identity/v3/projects/5ca4b7a74c924e6b80e3bd08e8bdfc97 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:47.035077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.036355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.069216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.070158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.070650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.071104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.072124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.074138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.092017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2136] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:47.092338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.094578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.109336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25a460-2ddc-4ae4-998a-2d6a38827ee0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.114584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.115508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.124560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.124933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] PATH_INFO: `/v3/projects/e1dada5ebd6b443d8ca09f47b21da600/users/39fd98ab1b0b4a0fa8dc524587c77330/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.138740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1ab8d-3376-4f54-9cda-25e53e309daf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.142821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dada5ebd6b443d8ca09f47b21da600, user_id=39fd98ab1b0b4a0fa8dc524587c77330, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.143229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.144031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.144437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.144860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.145637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6769a-4295-48ce-b506-31c914695b34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.147802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.167911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2137] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:47 2022] PUT /identity/v3/projects/e1dada5ebd6b443d8ca09f47b21da600/users/39fd98ab1b0b4a0fa8dc524587c77330/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:47.169636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.171263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.178403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.178689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.178961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.185389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2138] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:47.215338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] MFA Rules not processed for user `39fd98ab1b0b4a0fa8dc524587c77330`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:47.216925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.288924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2139] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:47.293516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.303435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8e827-25cc-4165-a131-079fb5c95010 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.338574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:47.379570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.380463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.381089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.381178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.382145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.384275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.415843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2140] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:47 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:47.417025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.427850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f16f-2174-4dd0-a9ff-583c8f538080 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.457389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:47.493236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.493713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.494127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.494407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.495119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.497444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.524785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2141] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:47 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:47.528393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.540188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3c5ca-10b8-4d56-b76f-7c330735a2e1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.545518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.546193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.546841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.546959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.548007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.549998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.568836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2142] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:10:47.571269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.586421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504147cb-ae96-44f3-bfe8-381278e7dcbe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.592394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.593356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.593643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.593977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] PATH_INFO: `/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c/users/61a9ea50df7e437ca616306a4f0cb3ab/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.610139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92190aa6fdb43aebc8b064c0ced8b3c, user_id=61a9ea50df7e437ca616306a4f0cb3ab, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.613387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.628071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2143] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:47 2022] PUT /identity/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c/users/61a9ea50df7e437ca616306a4f0cb3ab/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:47.631996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.643378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdc040-aadb-41ed-b211-37e99849e8ed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.703214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.703900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.704188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.704497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.705944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.707549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebd4a2-cf3f-4206-98fd-07f2471a67e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.717904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2144] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:47.722752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.733177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c811691-11da-43d1-bfdd-beccbaf1f190 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.738615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.739528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.739811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.740092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] PATH_INFO: `/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c/users/61a9ea50df7e437ca616306a4f0cb3ab/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.742417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92190aa6fdb43aebc8b064c0ced8b3c, user_id=61a9ea50df7e437ca616306a4f0cb3ab, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.745131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824aa6a-88d2-4875-a50d-9387ff1a27a3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:47.757230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2145] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:47 2022] PUT /identity/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c/users/61a9ea50df7e437ca616306a4f0cb3ab/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:47.768637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.768931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.769211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.800233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] MFA Rules not processed for user `61a9ea50df7e437ca616306a4f0cb3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:47.801573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.824373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.824798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.825470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.861731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] MFA Rules not processed for user `4d0e5c4a49d84ab9ab4c2db7d09e6fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:47.863014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.887567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2146] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:47.901197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.902539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.933578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:47.949299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2147] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:47.960521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:47.961905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:47.971757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4e357-4d74-459b-997e-b57b40d42ac9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.975896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.976606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.976879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.977362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.978825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe2c18-0f8a-4a32-b363-87e9545e589c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:47.982613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:47.983262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:47.983545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:47.983822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:47.990162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:47.991608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.002963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2148] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:48.004155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:48.005829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.016468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2149] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:48.405436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:48.414920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e126-3739-461c-b0dd-5f86c3f9ef54 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:48.419595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a9ea50df7e437ca616306a4f0cb3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:48.420268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:48.420562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:48.420849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:48.421568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:48.422445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:48.424135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.436990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2150] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:48.441320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:48.452404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713a62e-13f8-42a1-bc9f-ff75c3d0a527 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:48.457217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a9ea50df7e437ca616306a4f0cb3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:48.457875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:48.458196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:48.458477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:48.459183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:48.461162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.487751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2151] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:48 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:48.491848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:48.502523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07b020-76b0-48ff-ad6c-fb4f2d607522 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:48.560301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a9ea50df7e437ca616306a4f0cb3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:48.560983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:48.561336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:48.561616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:48.562354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:48.562665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:48.564311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.576312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2152] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:48.612264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:48.623954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a9c35-3fc3-4ffe-9795-d07b6d06f003 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:48.634380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a9ea50df7e437ca616306a4f0cb3ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:48.635272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:48.635567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:48.635842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] PATH_INFO: `/v3/projects/896da458a62e4f61a33c71f08f6f5179` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:48.637148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorizing `identity:delete_project(project_id=896da458a62e4f61a33c71f08f6f5179)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:48.639158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:48.676498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:48.690819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896da458a62e4f61a33c71f08f6f5179', 'request_id': 'req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:48.696424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896da458a62e4f61a33c71f08f6f5179', 'request_id': 'req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:48.703127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896da458a62e4f61a33c71f08f6f5179', 'request_id': 'req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:48.718436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] The token cache is being invalidate because project 896da458a62e4f61a33c71f08f6f5179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:48.718729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:48.719434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0b5b5b6e-1b96-4573-8329-8c4e2c5f33ee tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:48.721132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2153] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:48 2022] DELETE /identity/v3/projects/896da458a62e4f61a33c71f08f6f5179 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:48.918105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:48.920042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:48.983947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3767d-3ae6-43b1-8b5a-165451cf3911 tempest-VolumeQuotaClassesTest-2059231318 tempest-VolumeQuotaClassesTest-2059231318-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:49.035249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:49.043204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:49.044101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:49.044389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:49.044704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] PATH_INFO: `/v3/users/39fd98ab1b0b4a0fa8dc524587c77330` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:49.045963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=39fd98ab1b0b4a0fa8dc524587c77330)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:49.047944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:49.098010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39fd98ab1b0b4a0fa8dc524587c77330'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.098701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39fd98ab1b0b4a0fa8dc524587c77330'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.099373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39fd98ab1b0b4a0fa8dc524587c77330'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.111615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39fd98ab1b0b4a0fa8dc524587c77330'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.119953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39fd98ab1b0b4a0fa8dc524587c77330'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.133260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2154] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:48 2022] DELETE /identity/v3/users/39fd98ab1b0b4a0fa8dc524587c77330 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:49.137908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a76f29-a368-40e2-a0b7-56b2b0a2f68e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:49.138197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a76f29-a368-40e2-a0b7-56b2b0a2f68e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:49.138477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a76f29-a368-40e2-a0b7-56b2b0a2f68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:49.156539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98a76f29-a368-40e2-a0b7-56b2b0a2f68e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:49.157703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98a76f29-a368-40e2-a0b7-56b2b0a2f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:49.218760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2155] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:49.228549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:49.230043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:49.291246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b77e-2c40-425f-89b8-25593a4357fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:49.295198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:49.295826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:49.296134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:49.296421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:49.306060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:49.307482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:49.317379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2156] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:10:49.450581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:49.451103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:49.451182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:49.469269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] MFA Rules not processed for user `4d0e5c4a49d84ab9ab4c2db7d09e6fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:49.470711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:49.533762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2157] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:49.840650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:49.842171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:49.858600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0aea-9f62-422c-9098-01d07b5d8d87 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:49.870646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:49.871528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:49.871811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:49.872403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] PATH_INFO: `/v3/projects/e1dada5ebd6b443d8ca09f47b21da600` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:49.873658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dada5ebd6b443d8ca09f47b21da600)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:49.875655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:49.924331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dada5ebd6b443d8ca09f47b21da600', 'request_id': 'req-ac6b4a47-18da-4608-a03b-390d0ac682da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.930085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dada5ebd6b443d8ca09f47b21da600', 'request_id': 'req-ac6b4a47-18da-4608-a03b-390d0ac682da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.938839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dada5ebd6b443d8ca09f47b21da600', 'request_id': 'req-ac6b4a47-18da-4608-a03b-390d0ac682da'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.946977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:49.951049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] The token cache is being invalidate because project e1dada5ebd6b443d8ca09f47b21da600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:49.951934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.953026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:49.956490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2158] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:49 2022] DELETE /identity/v3/projects/e1dada5ebd6b443d8ca09f47b21da600 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:49.959326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:49.970382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab74ef0-5550-4b5f-809b-d845b2c6598f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:50.029506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:50.030663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.030979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.031309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] PATH_INFO: `/v3/users/61a9ea50df7e437ca616306a4f0cb3ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.032636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a9ea50df7e437ca616306a4f0cb3ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:50.034656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:50.041423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:50.091439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a9ea50df7e437ca616306a4f0cb3ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.092189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a9ea50df7e437ca616306a4f0cb3ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.108839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a9ea50df7e437ca616306a4f0cb3ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.109981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a9ea50df7e437ca616306a4f0cb3ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.118161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-75de78e4-c611-4c31-9707-c99c82908076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a9ea50df7e437ca616306a4f0cb3ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.135126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2159] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:49 2022] DELETE /identity/v3/users/61a9ea50df7e437ca616306a4f0cb3ab => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:50.437572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:50.439070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:50.456911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b4a47-18da-4608-a03b-390d0ac682da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:50.544843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:50.546024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.546024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.546454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] PATH_INFO: `/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.547752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92190aa6fdb43aebc8b064c0ced8b3c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:50.550482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:50.604933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'request_id': 'req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.610776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'request_id': 'req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.621169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92190aa6fdb43aebc8b064c0ced8b3c', 'request_id': 'req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.634696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] The token cache is being invalidate because project f92190aa6fdb43aebc8b064c0ced8b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:50.635234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.635997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d93bb23-6f05-4478-9e2b-e3cd154a9c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:50.637602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2160] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:50 2022] DELETE /identity/v3/projects/f92190aa6fdb43aebc8b064c0ced8b3c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:50.644690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.645198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.645300 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.663792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:50.665067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:50.745786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2161] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:50.754284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.754577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.754862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.773958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:50.775244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:50.784233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2162] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:50.788709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:50.799399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8091d-b76d-4efa-b6aa-dd93c63f132a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:50.804946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:50.805619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.805906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.806177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.806922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:50.807233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:50.808896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:50.821431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2163] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:50.825869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:50.835258 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b011371-59f3-4cd3-8abb-7b43554a5a63 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:50.839948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:50.840634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.840925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.841199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.841914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:50.843831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:50.880207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2164] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:50 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:50.886135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:50.897144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40804c14-eae0-459d-a3c6-9e44ee805503 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:50.956838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:50.957509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:50.957796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:50.958081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:50.958768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:50.960747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:50.987269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2165] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:50 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:50.991520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.002749 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128087-5650-4091-903c-996814cff3d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.008077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.008897 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.009166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.009450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.010297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.011975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.024683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2166] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:10:51.028640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.034009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:51.039711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5c44-27b4-4721-ad3a-f46ca241bc83 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.045512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.046160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.046442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.046898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.047784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.049859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2760d9c-9c5b-4fd5-9a93-06949142b3d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.060766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2167] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:10:51.065620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.076257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05565-f738-4800-8f71-0d858f361932 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.081629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.082506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.082790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.083067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] PATH_INFO: `/v3/projects/145d455ad13246fbbd36a23a98c0126f/users/7b654b420c2f43148e4f6e84479874b6/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.100970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145d455ad13246fbbd36a23a98c0126f, user_id=7b654b420c2f43148e4f6e84479874b6, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.103617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.118916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2168] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:51 2022] PUT /identity/v3/projects/145d455ad13246fbbd36a23a98c0126f/users/7b654b420c2f43148e4f6e84479874b6/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:51.130978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.131270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.131552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.160555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] MFA Rules not processed for user `7b654b420c2f43148e4f6e84479874b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:51.161829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:51.259215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2169] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:51.263296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.307887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:51.347975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9d4df-800e-4af3-b07c-915f1e0871d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.353533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.354203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.354488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.354767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.355464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.357516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.385189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2170] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:51.389304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.400535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9566-d78f-4157-b695-3d9117f3255a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.457796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.458634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.458984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.459323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.460127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.462216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.488084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2171] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:51.492024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.505383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf395319-521d-4753-8a30-f0b5b2638e18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.513047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.514011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.515260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.515642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.516513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.519108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.533681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2172] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:10:51.534936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.545113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cae38-9f54-4fa6-82c6-dc32d34a12eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.549799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.550469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.550814 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.551100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.551830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.553517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715759ba-8fc8-4664-b7a2-579d8937e59a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.563640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2173] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:51.568137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.578383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997aaca-9257-45f0-95a3-3a4c26fdb549 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.583252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.584234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.584555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.584866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] PATH_INFO: `/v3/projects/94299ff859a3489c981446a76c4d5af3/users/6879607a3ee641bb8da919d5b7f7c3eb/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.601636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94299ff859a3489c981446a76c4d5af3, user_id=6879607a3ee641bb8da919d5b7f7c3eb, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.604481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.618807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2174] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:51 2022] PUT /identity/v3/projects/94299ff859a3489c981446a76c4d5af3/users/6879607a3ee641bb8da919d5b7f7c3eb/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:51.629774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.630113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.630397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.658607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] MFA Rules not processed for user `6879607a3ee641bb8da919d5b7f7c3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:51.659836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:51.712492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:51.734619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2175] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:51.739628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.814938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ca238-5bbb-4565-ae14-89ee2e2b1250 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.820306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.821043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.821336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.821618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.822332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.824503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.865210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2176] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:51.869025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.882256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569fc1-bce3-48cc-9b80-48bd12dda388 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:51.951795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:51.952511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:51.952803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:51.953085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:51.953784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:51.956302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:51.983160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2177] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:51 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:51.984719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:51.995502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a5fd5-3a9c-434e-8aa6-67b0995c754f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.000880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.001629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.002148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.002148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.003332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.004982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.017131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2178] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:52.021369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:52.031704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb112f-8a72-4f88-b610-503c54d9da1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.038268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.039624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.040119 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.040660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] PATH_INFO: `/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673/users/56db58b0195243afa585239036748f46/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.057580 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f21f44b44044579ffd6f1c3f0ad673, user_id=56db58b0195243afa585239036748f46, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.060261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.075712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2179] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:52 2022] PUT /identity/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673/users/56db58b0195243afa585239036748f46/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:52.079608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:52.089738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c41126-a4ff-4882-b278-b171fb670dc0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.139180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:52.151001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.152155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.152577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.152942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.153695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.155910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522544a0-35c7-47d7-9797-544283bbcfde admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.167514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2180] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:52.171653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:52.182902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21c9c-a4c5-4e10-ad9d-cb06165d8a8d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.187709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.188737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.189073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.189384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] PATH_INFO: `/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673/users/56db58b0195243afa585239036748f46/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.191354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f21f44b44044579ffd6f1c3f0ad673, user_id=56db58b0195243afa585239036748f46, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.194105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.204545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2181] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:52 2022] PUT /identity/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673/users/56db58b0195243afa585239036748f46/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:52.214952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.215348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.215736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.243661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] MFA Rules not processed for user `56db58b0195243afa585239036748f46`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:52.245031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:52.324735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2182] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:52.337257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:52.339203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:52.405032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85fe16-7f50-4fb0-abf8-407f53de43e9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.409678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.410369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.410660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.410939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.421126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.422584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.433305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2183] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:52.475656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:52.477128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:52.490628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf6636-6de6-45f9-bd5f-758ee2720517 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:52.494456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:52.495116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:52.495406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:52.495695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:52.571590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:52.573106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa6b1d-1fa7-4ac1-9274-b501c1d80fe5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:52.583061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2184] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:56.578840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:56.624524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:56.642375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d29c-ca3c-493b-8957-12a4d0100bd8 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:56.649859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:56.650755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:56.651096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:56.651365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] PATH_INFO: `/v3/users/7b654b420c2f43148e4f6e84479874b6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:56.652607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b654b420c2f43148e4f6e84479874b6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:56.654566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:56.707109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b654b420c2f43148e4f6e84479874b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:56.707930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b654b420c2f43148e4f6e84479874b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:56.721550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b654b420c2f43148e4f6e84479874b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:56.722325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b654b420c2f43148e4f6e84479874b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:56.729950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b654b420c2f43148e4f6e84479874b6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:56.748027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2185] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:56 2022] DELETE /identity/v3/users/7b654b420c2f43148e4f6e84479874b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:56.752227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:56.752784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:56.752784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:56.769950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:56.771314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:56.829992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2186] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:56.839798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:56.841351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:56.910556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239950-9c7f-41d9-abc4-c4845c919a9c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:56.915035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:56.915707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:56.915979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:56.916247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:56.926460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:56.927912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:56.937554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2187] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:10:57.325461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:57.327127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:57.340316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896f525-1cde-4fff-9134-3e7a48c45c14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:57.345320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:57.346248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:57.346541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:57.347014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] PATH_INFO: `/v3/projects/145d455ad13246fbbd36a23a98c0126f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:57.348428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145d455ad13246fbbd36a23a98c0126f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:57.350536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:57.357430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:57.404607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145d455ad13246fbbd36a23a98c0126f', 'request_id': 'req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.412980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145d455ad13246fbbd36a23a98c0126f', 'request_id': 'req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.425092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145d455ad13246fbbd36a23a98c0126f', 'request_id': 'req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.436808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] The token cache is being invalidate because project 145d455ad13246fbbd36a23a98c0126f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:57.437096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.437815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dd2e82ce-ffd8-4ae1-a92d-acb70a31d416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.439528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2188] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:57 2022] DELETE /identity/v3/projects/145d455ad13246fbbd36a23a98c0126f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:57.443752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:57.453561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89a68e-546c-4256-9cd5-56aab045541b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:57.513708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:57.514590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:57.514867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:57.515135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] PATH_INFO: `/v3/users/6879607a3ee641bb8da919d5b7f7c3eb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:57.516337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6879607a3ee641bb8da919d5b7f7c3eb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:57.518353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:57.523880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:57.567521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6879607a3ee641bb8da919d5b7f7c3eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.568098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6879607a3ee641bb8da919d5b7f7c3eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.580745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6879607a3ee641bb8da919d5b7f7c3eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.581567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6879607a3ee641bb8da919d5b7f7c3eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.589212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6879607a3ee641bb8da919d5b7f7c3eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:57.603434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2189] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:57 2022] DELETE /identity/v3/users/6879607a3ee641bb8da919d5b7f7c3eb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:57.927460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:57.927966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:57.928913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:57.935471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:57.937133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:57.960542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] MFA Rules not processed for user `41c4d3bab2164767b8eefcb4427b65b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:57.961827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:58.019706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff281fde-b8e6-46a9-b785-15a9df3c8cdd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:58.024440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:58.025410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:58.026006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:58.026006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] PATH_INFO: `/v3/projects/94299ff859a3489c981446a76c4d5af3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:58.027322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94299ff859a3489c981446a76c4d5af3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:58.029401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:58.040865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2190] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:58.081989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94299ff859a3489c981446a76c4d5af3', 'request_id': 'req-6c4c10a3-f81d-446d-b015-51277fefe528'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.089024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94299ff859a3489c981446a76c4d5af3', 'request_id': 'req-6c4c10a3-f81d-446d-b015-51277fefe528'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.096880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94299ff859a3489c981446a76c4d5af3', 'request_id': 'req-6c4c10a3-f81d-446d-b015-51277fefe528'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.125321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] The token cache is being invalidate because project 94299ff859a3489c981446a76c4d5af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:58.125872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.126642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.128595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2191] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:57 2022] DELETE /identity/v3/projects/94299ff859a3489c981446a76c4d5af3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:10:58.132477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:58.143123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc784-54ea-4c38-b5b9-796b1c068609 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:58.204430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:58.205435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:58.205809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:58.206211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] PATH_INFO: `/v3/users/56db58b0195243afa585239036748f46` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:58.207539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=56db58b0195243afa585239036748f46)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:58.209606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:58.255454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:58.261247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56db58b0195243afa585239036748f46'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.262282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56db58b0195243afa585239036748f46'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.262977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56db58b0195243afa585239036748f46'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.276385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56db58b0195243afa585239036748f46'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.285364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bfcff94b-89c0-48ab-bbd1-0dad99ecf9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56db58b0195243afa585239036748f46'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.301002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2192] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:10:58 2022] DELETE /identity/v3/users/56db58b0195243afa585239036748f46 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:58.757275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:58.759405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:58.774982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c10a3-f81d-446d-b015-51277fefe528 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:58.787559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:58.846110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:58.847045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:58.847327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:58.847605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] PATH_INFO: `/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:58.848990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f21f44b44044579ffd6f1c3f0ad673)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:58.850935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:58.909429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f21f44b44044579ffd6f1c3f0ad673', 'request_id': 'req-e596b438-b8b0-4515-8ed0-3ff20617e4df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.922480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f21f44b44044579ffd6f1c3f0ad673', 'request_id': 'req-e596b438-b8b0-4515-8ed0-3ff20617e4df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.929813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f21f44b44044579ffd6f1c3f0ad673', 'request_id': 'req-e596b438-b8b0-4515-8ed0-3ff20617e4df'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.953939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] The token cache is being invalidate because project c6f21f44b44044579ffd6f1c3f0ad673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:10:58.954443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.955289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e596b438-b8b0-4515-8ed0-3ff20617e4df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:10:58.957643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2193] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:10:58 2022] DELETE /identity/v3/projects/c6f21f44b44044579ffd6f1c3f0ad673 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:58.963896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:58.964460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:58.965438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:58.989792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:58.991311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:59.065424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2194] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:59.074664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.075027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.075381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.095015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:59.096976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:59.105495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2195] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:10:59.109823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.119870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae220d26-5e6b-4033-8d46-f6a3c79ec963 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.125667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.126334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.126617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.126895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.127628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:10:59.127979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.129799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.146666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2196] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:10:59.147517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.157438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f41d19-6a4b-444f-ad95-c87c795b542d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.163164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.164182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.164655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.165223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.165976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.168052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.198929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2197] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:10:59 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:10:59.204161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.215731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925166-db9f-4c83-8d72-eeec537fd999 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.292494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.293285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.293508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.293785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.294504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.296519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.324401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2198] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:10:59 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:10:59.328444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.340443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7d4db-a28b-4033-a665-ebf1469965e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.345007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.345668 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.345951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.346227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.347289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.348942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.360011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2199] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:59.364406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.379908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c74f45-29fb-4474-9119-613235289ee9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.385265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.385924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.386211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.386517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.387243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.388929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a233c03-7971-493f-9b25-6e7d118d0205 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.400135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2200] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:10:59.404467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.418935 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303668e-6348-4eca-bfad-d5fd045de6a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.430323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.431773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.432194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.432669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] PATH_INFO: `/v3/projects/e869fe2d1c864e0d89d2ac27597f8f83/users/d5de255df1f048d0b92d3ead4448dfff/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.452352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e869fe2d1c864e0d89d2ac27597f8f83, user_id=d5de255df1f048d0b92d3ead4448dfff, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.455081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.473211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2201] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:10:59 2022] PUT /identity/v3/projects/e869fe2d1c864e0d89d2ac27597f8f83/users/d5de255df1f048d0b92d3ead4448dfff/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:10:59.478095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16814-8fd4-433c-92af-c9ed9b5a1ec7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.478393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16814-8fd4-433c-92af-c9ed9b5a1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.478680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a16814-8fd4-433c-92af-c9ed9b5a1ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.497819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1a16814-8fd4-433c-92af-c9ed9b5a1ec7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:10:59.498896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1a16814-8fd4-433c-92af-c9ed9b5a1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:59.521343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:59.577661 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2202] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:10:59.587665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:10:59.589536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:10:59.660202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:10:59.678625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2250f-82eb-4c24-8814-e4158706b80c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:10:59.684221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:10:59.684942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:10:59.685230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:10:59.685512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:10:59.697005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:10:59.698514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:10:59.710394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2203] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:00.440720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.441395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.441395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.477000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] MFA Rules not processed for user `d5de255df1f048d0b92d3ead4448dfff`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:00.478556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:00.561849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2204] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:00.566492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.576230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa80c-ced3-4253-8d3c-50c601b0b4cf service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.581180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.581834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.582118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.582395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.583105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.585086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.617355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2205] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:11:00 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:11:00.621247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.633919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc8286-3ca2-47ad-bd4f-0318726a0002 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.703657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.704352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.704671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.704986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.705694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.707827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.736579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2206] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:11:00 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:11:00.740702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.754201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c890fc-15c5-4e80-bda5-41c4ae5f1707 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.759387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.760251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.760722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.761009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.761984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.765098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.778766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2207] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:11:00.783193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.794079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962b3a-83a3-429e-a7ea-3e38c166b879 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.798903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.799858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.800136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.800441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] PATH_INFO: `/v3/projects/5c20d35cbd114ca6b48bc962f9275073/users/3ff4e6b7c6404f41a11ddf59f55c3cd4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.817908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c20d35cbd114ca6b48bc962f9275073, user_id=3ff4e6b7c6404f41a11ddf59f55c3cd4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.820646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.846352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2208] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:11:00 2022] PUT /identity/v3/projects/5c20d35cbd114ca6b48bc962f9275073/users/3ff4e6b7c6404f41a11ddf59f55c3cd4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:00.850346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.861784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429fc42-d3a2-4446-a4e7-268b690527e3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.931096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.931704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.931991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.932267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.933111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.934916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ce09d-7327-4e18-bfaa-48da1a3c22cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.949695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2209] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:11:00.954116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:00.965640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b1ff2-5044-413a-995d-233e943fee63 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:00.970838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:00.971798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:00.972080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:00.972353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] PATH_INFO: `/v3/projects/5c20d35cbd114ca6b48bc962f9275073/users/3ff4e6b7c6404f41a11ddf59f55c3cd4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:00.974397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c20d35cbd114ca6b48bc962f9275073, user_id=3ff4e6b7c6404f41a11ddf59f55c3cd4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:00.977125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:00.991603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2210] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:11:00 2022] PUT /identity/v3/projects/5c20d35cbd114ca6b48bc962f9275073/users/3ff4e6b7c6404f41a11ddf59f55c3cd4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:01.678552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:01.679073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:01.679155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:01.712313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] MFA Rules not processed for user `3ff4e6b7c6404f41a11ddf59f55c3cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:01.713724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:01.824837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2211] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:01.832871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:01.836686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:01.850623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:01.921942 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a095df7-a929-404d-8f16-9a842a691299 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:01.926295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:01.927076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:01.927421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:01.927764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:01.989891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:01.991423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30a95-dd77-4d1b-b2de-7bd801fc3962 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:02.001311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:11:03.365568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:03.370909 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e08899-bbe6-4887-b02e-5c49e256a3c4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:03.375653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:03.376346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:03.376748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:03.376946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:03.388714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:03.390387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:03.401126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:11.997463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=45 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:13.397102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:14.645932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:14.646430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:14.647068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:14.666875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] MFA Rules not processed for user `8bd2fc9f3216425cbcbbed84b7815529`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:14.668142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:14.727766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2214] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:16.960794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:17.097011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3e4bb-a8fc-4e6d-ac40-02b528bdc965 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:17.103254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:17.104260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:17.104582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:17.105071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] PATH_INFO: `/v3/users/d5de255df1f048d0b92d3ead4448dfff` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:17.113026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5de255df1f048d0b92d3ead4448dfff)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:17.115427 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:17.192104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5de255df1f048d0b92d3ead4448dfff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.193009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5de255df1f048d0b92d3ead4448dfff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.204976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5de255df1f048d0b92d3ead4448dfff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.205645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5de255df1f048d0b92d3ead4448dfff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.213564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5de255df1f048d0b92d3ead4448dfff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.231736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2215] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:11:16 2022] DELETE /identity/v3/users/d5de255df1f048d0b92d3ead4448dfff => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:11:17.598691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:17.652951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bafb0-e8cc-45bd-8239-13497d566e09 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:17.734794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:17.735709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:17.736458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:17.736773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] PATH_INFO: `/v3/projects/e869fe2d1c864e0d89d2ac27597f8f83` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:17.738179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e869fe2d1c864e0d89d2ac27597f8f83)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:17.742208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:17.805228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e869fe2d1c864e0d89d2ac27597f8f83', 'request_id': 'req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.812890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e869fe2d1c864e0d89d2ac27597f8f83', 'request_id': 'req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.821803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e869fe2d1c864e0d89d2ac27597f8f83', 'request_id': 'req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.853827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] The token cache is being invalidate because project e869fe2d1c864e0d89d2ac27597f8f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:11:17.854124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.854757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:17.856335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2216] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:11:17 2022] DELETE /identity/v3/projects/e869fe2d1c864e0d89d2ac27597f8f83 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:17.860341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:17.861976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:17.889687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa9013-0830-4516-8eb3-56506304481b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:17.928520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:17.981684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:17.982732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:17.983053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:17.983327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] PATH_INFO: `/v3/users/3ff4e6b7c6404f41a11ddf59f55c3cd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:17.984629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff4e6b7c6404f41a11ddf59f55c3cd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:17.986662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:18.064358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff4e6b7c6404f41a11ddf59f55c3cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.065174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff4e6b7c6404f41a11ddf59f55c3cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.082860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff4e6b7c6404f41a11ddf59f55c3cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.083777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff4e6b7c6404f41a11ddf59f55c3cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.091705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5835b1e-8575-4e25-a74e-4d532b983681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff4e6b7c6404f41a11ddf59f55c3cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.109057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2217] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:11:17 2022] DELETE /identity/v3/users/3ff4e6b7c6404f41a11ddf59f55c3cd4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:18.308788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:18.310922 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:18.328327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac9bc-846a-497f-a8cb-128ae0eaf7e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:18.331792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:18.406379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:18.407298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.407600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.407842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] PATH_INFO: `/v3/projects/5c20d35cbd114ca6b48bc962f9275073` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.409122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c20d35cbd114ca6b48bc962f9275073)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:18.411045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:18.466695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c20d35cbd114ca6b48bc962f9275073', 'request_id': 'req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.474193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c20d35cbd114ca6b48bc962f9275073', 'request_id': 'req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.483566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c20d35cbd114ca6b48bc962f9275073', 'request_id': 'req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.499709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] The token cache is being invalidate because project 5c20d35cbd114ca6b48bc962f9275073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:11:18.499970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.500789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a42b5c2-4a3c-46f8-9beb-8f9cdf90ecad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:18.502792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2218] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:11:18 2022] DELETE /identity/v3/projects/5c20d35cbd114ca6b48bc962f9275073 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:18.508965 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.509337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.509681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.577113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:18.578632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:18.658207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2219] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:18.666108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.666590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.666781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.686988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:18.688203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:18.697359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2220] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:18.701639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:18.711952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a39a-7e19-4e96-b704-bcb91ff2e158 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:18.722664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:18.723285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.724266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.724266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.725694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:11:18.725694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:18.727285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:18.756988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2221] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:11:18.761363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:18.775102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b9d87-2095-4e12-ac02-c250f5b6cf76 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:18.780532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:18.781554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.782185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.783721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.785057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:18.787389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:18.828755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2222] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:11:18 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:11:18.832792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:18.844501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea675c-1dec-4739-8532-50ba2ba3c2bd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:18.856650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:18.929449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:18.930178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.930468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.930744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.931455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:18.933464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:18.964152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2223] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:11:18 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:11:18.969294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:18.979849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba25c2b-65db-4325-b407-f527a9698979 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:18.984391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:18.985079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:18.985364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:18.985636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:18.986363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:18.987968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.000962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2224] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:11:19.014325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:19.025085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464fdca-4f8f-49b2-a57a-771029f954b7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:19.030089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:19.030770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.031169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.031328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.032108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:19.033784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5140-dae1-4853-8c5b-b0373058e356 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.045139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2225] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:11:19.049309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:19.060068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a259e-3a48-49a8-9328-5b5b80065584 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:19.064855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:19.065746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.066060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.066994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] PATH_INFO: `/v3/projects/c86f24239e124bde9e97897f27deac29/users/feec8f2998f14181b78ef0fce3812ca2/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.082721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86f24239e124bde9e97897f27deac29, user_id=feec8f2998f14181b78ef0fce3812ca2, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:19.085623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.100337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2226] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:11:19 2022] PUT /identity/v3/projects/c86f24239e124bde9e97897f27deac29/users/feec8f2998f14181b78ef0fce3812ca2/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:11:19.110554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.110943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.111293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.139492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] MFA Rules not processed for user `feec8f2998f14181b78ef0fce3812ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:19.140874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:19.217111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2227] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:19.229376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:19.231472 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:19.280520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:19.325815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1ef82-da4b-4b86-9a35-bebdd43495f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:19.330529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:19.331275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.331820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.332327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.343345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:19.345014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cffcf-81ac-4fa4-a825-0a053a0a881d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.356026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:19.377840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:19.460040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061cce6-fef3-475d-8171-2de22b8ab9c9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:19.465151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:19.466263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.466556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.466837 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] PATH_INFO: `/v3/users/feec8f2998f14181b78ef0fce3812ca2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.468142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] RBAC: Authorizing `identity:delete_user(user_id=feec8f2998f14181b78ef0fce3812ca2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:19.470139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.498875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:19.531198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feec8f2998f14181b78ef0fce3812ca2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:19.531967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feec8f2998f14181b78ef0fce3812ca2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:19.545212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feec8f2998f14181b78ef0fce3812ca2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:19.545922 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feec8f2998f14181b78ef0fce3812ca2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:19.554135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feec8f2998f14181b78ef0fce3812ca2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:19.571396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2229] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:11:19 2022] DELETE /identity/v3/users/feec8f2998f14181b78ef0fce3812ca2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:19.575974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.576655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.576655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.594349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:19.595500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:19.664771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2230] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:19.675578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:19.677995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:19.762281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a1669-7989-4122-95c7-0590233a605b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:19.766128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:19.766789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:19.767078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:19.767356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:19.777070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:19.778557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cb4d5-5929-48f6-b0d6-a17f668a0315 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:19.788550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:20.069899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.071851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:20.084193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95127a-e906-4c57-8b6b-bff1f8dd49c3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.089599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.090540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.090822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.091094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] PATH_INFO: `/v3/projects/c86f24239e124bde9e97897f27deac29` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.092361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86f24239e124bde9e97897f27deac29)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.094505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.152269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86f24239e124bde9e97897f27deac29', 'request_id': 'req-c2652b92-f507-4727-b4d2-e58759abaeea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:20.158619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86f24239e124bde9e97897f27deac29', 'request_id': 'req-c2652b92-f507-4727-b4d2-e58759abaeea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:20.169167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86f24239e124bde9e97897f27deac29', 'request_id': 'req-c2652b92-f507-4727-b4d2-e58759abaeea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:20.181350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] The token cache is being invalidate because project c86f24239e124bde9e97897f27deac29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:11:20.181646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:20.182320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2652b92-f507-4727-b4d2-e58759abaeea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:11:20.184057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2232] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:11:20 2022] DELETE /identity/v3/projects/c86f24239e124bde9e97897f27deac29 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:20.189485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.190047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.190047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.208796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:20.210329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:20.272151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2233] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:20.280802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.281095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.281375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.299950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:20.301192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:20.309451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2234] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:20.313579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.324563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade74c79-e096-4ca1-b7b1-f813992509fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.330894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.331556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.331839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.332116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.332895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:11:20.333533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.335167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.347529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2235] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:11:20.352588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.361940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771186a-4bcf-48bc-84b0-c0fecc6294ec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.366956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.367603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.367887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.368168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.368914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.370855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.398411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2236] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:11:20 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:11:20.402558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.413489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654449-5305-4cfe-85f3-38888d316d7a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.421932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:20.479137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.479820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.480106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.480389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.481106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.483053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.508484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2237] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:11:20 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:11:20.510255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.521935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42af27-71f0-40d8-9d8b-e5134d2ed52b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.527659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.528325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.528739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.528993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.529865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.531577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.543991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2238] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:11:20.548080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.558625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05cff-fcb3-42c3-8ed0-c35bb966282f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.563725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.564393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.564709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.564986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.565718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.567333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7e61e-48ad-4ae3-8872-28297744f7a6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.577820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2239] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:11:20.582015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.588959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:20.593061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3884a2-d7dc-4108-86df-75612f72c5bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.599240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.600251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.600786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.601231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] PATH_INFO: `/v3/projects/e5575685b576400aa11d301437487686/users/403308af98fd4c38a8f9177d45f1e4d5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.616067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5575685b576400aa11d301437487686, user_id=403308af98fd4c38a8f9177d45f1e4d5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.618832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.634837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2240] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:11:20 2022] PUT /identity/v3/projects/e5575685b576400aa11d301437487686/users/403308af98fd4c38a8f9177d45f1e4d5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:11:20.645349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.645713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.645994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.676083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] MFA Rules not processed for user `403308af98fd4c38a8f9177d45f1e4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:11:20.677504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:20.748978 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2241] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:11:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:11:20.760570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:20.762732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:20.825513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa40f0-c351-4d4f-91e9-9552b44ccfed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:20.832330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:20.833644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:20.833867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:20.834099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:20.843498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:20.844974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:20.854309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:26.779968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:26.782380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:26.842169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b173b-f94b-419c-8344-9f2f5b5a05d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:26.846467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:26.847116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:26.847403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:26.847673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:26.849749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:26.885386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:26.918272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:26.919764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78c74b-81de-4f55-90ea-d9733b7532d4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:26.932175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2243] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:11:30.849819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:36.925696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=14 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:11:51.906853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:11:51.908373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:52.007627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53faa5-4376-4850-89ff-54354bd0f552 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:11:52.014321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:11:52.015047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:11:52.015419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:11:52.015783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:11:52.017811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:11:52.030808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:11:52.032430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:11:52.044163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2244] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:11:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:12:02.039235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:12:28.391201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782877ef-016b-4a24-b910-1277f1f349f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:28.391741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782877ef-016b-4a24-b910-1277f1f349f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:28.391836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782877ef-016b-4a24-b910-1277f1f349f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:28.410683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-782877ef-016b-4a24-b910-1277f1f349f9 None None] MFA Rules not processed for user `4d0e5c4a49d84ab9ab4c2db7d09e6fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:12:28.411904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-782877ef-016b-4a24-b910-1277f1f349f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:12:28.478053 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2245] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:12:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:12:29.863973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:12:29.928225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73090901-1034-4fc1-b82e-7d034acce2aa service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:12:29.933034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:12:29.933910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:29.934189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:29.934463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] PATH_INFO: `/v3/users/403308af98fd4c38a8f9177d45f1e4d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:29.935663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] RBAC: Authorizing `identity:delete_user(user_id=403308af98fd4c38a8f9177d45f1e4d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:12:29.937619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:12:29.990200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403308af98fd4c38a8f9177d45f1e4d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:29.990848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403308af98fd4c38a8f9177d45f1e4d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:29.991447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403308af98fd4c38a8f9177d45f1e4d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.004471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403308af98fd4c38a8f9177d45f1e4d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.013167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403308af98fd4c38a8f9177d45f1e4d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.025862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2246] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:12:29 2022] DELETE /identity/v3/users/403308af98fd4c38a8f9177d45f1e4d5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:12:30.030360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:30.030852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:30.030947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:30.049689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:12:30.050929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:12:30.115363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2247] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:12:30.125952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:12:30.133582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:12:30.212623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba42f3-3202-48e0-aca5-f08d1159559b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:12:30.218203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:12:30.219172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:30.219575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:30.219949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:30.231290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:12:30.232761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e9d25-f0a2-4759-8ac6-ea27603a2310 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:12:30.246306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:12:30.680475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:12:30.682593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:12:30.696939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc56bf-1db6-40df-939d-bd391a408012 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:12:30.703309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:12:30.705005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:30.705289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:30.705632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] PATH_INFO: `/v3/projects/e5575685b576400aa11d301437487686` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:30.707310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5575685b576400aa11d301437487686)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:12:30.709492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:12:30.786558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5575685b576400aa11d301437487686', 'request_id': 'req-2ee25b25-90da-497e-a9e1-810fa4a64d10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.793268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5575685b576400aa11d301437487686', 'request_id': 'req-2ee25b25-90da-497e-a9e1-810fa4a64d10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.801576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5575685b576400aa11d301437487686', 'request_id': 'req-2ee25b25-90da-497e-a9e1-810fa4a64d10'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.827974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] The token cache is being invalidate because project e5575685b576400aa11d301437487686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:12:30.828343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.829081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2ee25b25-90da-497e-a9e1-810fa4a64d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:12:30.830852 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2249] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:12:30 2022] DELETE /identity/v3/projects/e5575685b576400aa11d301437487686 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:12:39.920804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:12:39.921335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:12:39.921433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:12:39.938994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] MFA Rules not processed for user `5bb9100dcf2a4522840d726d800fe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:12:39.940316 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:12:39.957933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:12:39.997809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2250] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:12:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:12:40.814843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:12:49.990639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=15 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:00.682611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e1db-5ebb-4657-b4ec-8e0415fffb86 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:00.683138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e1db-5ebb-4657-b4ec-8e0415fffb86 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:00.683259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e1db-5ebb-4657-b4ec-8e0415fffb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:00.749170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bfb7e1db-5ebb-4657-b4ec-8e0415fffb86 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:00.750397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bfb7e1db-5ebb-4657-b4ec-8e0415fffb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:00.838948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2251] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:00.888288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:00.889994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:00.993126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6da70-3720-497f-b111-d3f48aa6e004 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:00.998770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:00.999410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:00.999724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:00.999984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:01.009150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:01.010821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed71d4-e180-43f4-bf69-17931ab3f53b service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:01.024485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2252] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:13:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:13:10.835322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=47 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:11.016236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:18.008644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.009238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.009238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.010636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2253] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:13:18.016320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511257c8-37c3-4dc2-b7c0-0c1dc4cd58a4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.016788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511257c8-37c3-4dc2-b7c0-0c1dc4cd58a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.016903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511257c8-37c3-4dc2-b7c0-0c1dc4cd58a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.076077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-511257c8-37c3-4dc2-b7c0-0c1dc4cd58a4 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:18.077338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-511257c8-37c3-4dc2-b7c0-0c1dc4cd58a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.156059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2254] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:13:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:18.171894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:18.174552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.261366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123efaa-32eb-4f60-9c2b-5d08d22ead2a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:18.265560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:18.266240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.266663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.266946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.278893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:18.280384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707729a-2968-4f1c-ae26-d326efb1116d service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:18.294900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2255] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:13:18.368201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.368551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.368843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.370251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2256] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:13:18.375832 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd792-ad1d-4fff-a8e2-db319df21a96 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.376121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd792-ad1d-4fff-a8e2-db319df21a96 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.376400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd792-ad1d-4fff-a8e2-db319df21a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.395018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5cdbd792-ad1d-4fff-a8e2-db319df21a96 None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:18.396245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5cdbd792-ad1d-4fff-a8e2-db319df21a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.449933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2257] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:13:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:18.462088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:18.463945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.479083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b781e-6bfc-40d3-84fe-c65ce9cdf453 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:18.483038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:18.483575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.483862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.484141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.491792 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:18.493265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b48c5e-08cd-4905-835d-7fde05c6a21a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:18.502401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:13:18.584270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.584597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.584881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.586337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2259] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 03:13:18.591868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424726d2-f2c5-451c-824e-5d033e1874f4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.592180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424726d2-f2c5-451c-824e-5d033e1874f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.592495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424726d2-f2c5-451c-824e-5d033e1874f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.612546 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-424726d2-f2c5-451c-824e-5d033e1874f4 None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:18.613773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-424726d2-f2c5-451c-824e-5d033e1874f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.629070 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2260] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:13:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:18.640176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:18.642360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:18.660512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96fc9-1e0a-4852-a084-4c8ba52bee3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:18.664987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:18.665644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:18.665932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:18.666206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:18.678778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:18.680226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:18.691490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:13:23.798718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:23.799248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:23.799248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:23.819091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:23.820327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:23.828013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2262] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:13:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:23.841842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:23.853788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f69130-1623-47ae-8c18-1059553a50c7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:23.859752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:23.860452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:23.860796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:23.861077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:23.869898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:23.871508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:23.880531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2263] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:13:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:13:33.813120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=62 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:33.876913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=70 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:41.872547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:41.882866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75e57-4164-4804-a8e8-95d60e62b813 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:41.887230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:41.887871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:41.888160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:41.888437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:41.890542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:41.905171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:41.906578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:41.918469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2264] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:13:45.500401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:45.511878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80383868-6fba-4df3-bb63-ad8f8981e043 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:45.516588 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:45.517232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:45.517520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:45.517842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:45.520552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:45.537021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:45.539199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:45.556834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:13:51.913038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:55.548859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:56.830717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:56.832633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:56.906586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003744e6-e8c4-4d30-8775-ca83405a6728 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:56.912102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190969-5172-4790-b03f-5831107de16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:56.913013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190969-5172-4790-b03f-5831107de16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:56.913303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190969-5172-4790-b03f-5831107de16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:56.913580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190969-5172-4790-b03f-5831107de16c admin admin] PATH_INFO: `/v3/users/4d0e5c4a49d84ab9ab4c2db7d09e6fac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:56.914887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190969-5172-4790-b03f-5831107de16c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0e5c4a49d84ab9ab4c2db7d09e6fac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:56.916874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190969-5172-4790-b03f-5831107de16c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:56.969422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-15190969-5172-4790-b03f-5831107de16c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0e5c4a49d84ab9ab4c2db7d09e6fac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:56.970158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-15190969-5172-4790-b03f-5831107de16c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0e5c4a49d84ab9ab4c2db7d09e6fac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:56.970902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-15190969-5172-4790-b03f-5831107de16c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0e5c4a49d84ab9ab4c2db7d09e6fac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:56.983353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-15190969-5172-4790-b03f-5831107de16c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0e5c4a49d84ab9ab4c2db7d09e6fac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:56.992210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-15190969-5172-4790-b03f-5831107de16c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0e5c4a49d84ab9ab4c2db7d09e6fac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.005849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2266] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:13:56 2022] DELETE /identity/v3/users/4d0e5c4a49d84ab9ab4c2db7d09e6fac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:13:57.193558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.204686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567efc8c-906d-4bd8-893f-75fcc8b2f730 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.266123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.267002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.267284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.267559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] PATH_INFO: `/v3/projects/0d8e2178a3c84520ac00fc1cf2da0329` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.269351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8e2178a3c84520ac00fc1cf2da0329)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.271295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.320186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8e2178a3c84520ac00fc1cf2da0329', 'request_id': 'req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.325854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8e2178a3c84520ac00fc1cf2da0329', 'request_id': 'req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.332554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8e2178a3c84520ac00fc1cf2da0329', 'request_id': 'req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.348473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] The token cache is being invalidate because project 0d8e2178a3c84520ac00fc1cf2da0329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:13:57.348765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.349405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0f8ccff0-afc0-4bee-b576-f06b8ad8d7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:13:57.350959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2267] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:13:57 2022] DELETE /identity/v3/projects/0d8e2178a3c84520ac00fc1cf2da0329 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:13:57.356020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.356309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.356618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.379595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:57.380861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:57.458327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2268] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:57.465980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.466263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.466536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.485806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:57.487139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:57.496283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2269] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:57.501146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.511511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23309987-3dbd-4673-b8b5-00b2d552ee9b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.516486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.517204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.517483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.517756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.518480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:13:57.518774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.520451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.533583 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2270] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:13:57.538408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.548344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f89aff-2012-44a8-ad41-858508efeb97 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.553751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.554434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.554718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.554989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.555698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.557688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.586448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2271] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:13:57 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:13:57.590844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.601935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0664e73-8eda-4ce2-8d84-814ad8e774ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.606340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:57.668271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.669013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.669294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.669573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.670269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.672218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.695848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2272] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:13:57 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:13:57.697336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.708306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71cd10-bf08-41eb-b6ff-60adf6891736 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.714186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.714903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.715192 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.715451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.716195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.717889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.732195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2273] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:13:57.736537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.748672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c411f-08a7-4543-ba9e-6327438437aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.754804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.755834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.756259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.756773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.758125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.760750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71aa6a-87cb-45a1-9176-f2498723e737 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.780305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2274] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:13:57.784071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.794807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bebc5b-feac-4b08-a33e-1646b410e8bb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:57.800159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:57.801092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.801377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.801655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] PATH_INFO: `/v3/projects/4fc64c9b7cd84031b153d4548ea6d810/users/d2c60feea143457c9e2c540e5c7c63b7/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.815798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc64c9b7cd84031b153d4548ea6d810, user_id=d2c60feea143457c9e2c540e5c7c63b7, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:57.818476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:57.834150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2275] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:13:57 2022] PUT /identity/v3/projects/4fc64c9b7cd84031b153d4548ea6d810/users/d2c60feea143457c9e2c540e5c7c63b7/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:13:57.844315 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:57.844833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:57.844946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:57.871459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] MFA Rules not processed for user `d2c60feea143457c9e2c540e5c7c63b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:13:57.873020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:57.941079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2276] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:13:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:13:57.954276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:57.956201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:57.988475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:58.042227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e155d33-a68a-481d-a41c-412e6ddc6c8d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:58.046238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:58.046915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:58.047200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:58.047477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:58.056214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:58.057711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edafa9-30e9-4fbf-b73b-eca0f7bdb7ed service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:58.067438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2277] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:13:58.800186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:13:58.802037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:58.885123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078bef0-b9a4-46dc-88f8-9645e01066ec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:13:58.891693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:13:58.892351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:13:58.892686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:13:58.892966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:13:58.894936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:13:58.906123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:13:58.967156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:13:58.968827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:13:58.978218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2278] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:13:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:14:04.732384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:04.732865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:04.732963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:04.752374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:14:04.753642 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:04.765562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:04.767171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:04.783513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c128b-309a-446f-89bc-619ad531f533 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:04.790584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:04.791260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:04.791545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:04.791820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:04.793924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:04.814577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2279] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:14:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:14:04.837227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:04.838677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:04.856081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:04.857744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:04.868043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:14:04.912783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0b06-b113-4526-ab3b-2c37268c475f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:04.921206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:04.921869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:04.922152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:04.922424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:04.933816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:04.935416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06965c-834b-43e6-bf46-8b9892034a02 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:04.945750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2281] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:14:07.260572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:07.342198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a8926-3e8c-4a69-bf78-428b90ae3e05 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:07.348738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:07.349617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:07.349902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:07.350173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] PATH_INFO: `/v3/users/d2c60feea143457c9e2c540e5c7c63b7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:07.351565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c60feea143457c9e2c540e5c7c63b7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:07.353765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:07.407239 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c60feea143457c9e2c540e5c7c63b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.407915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c60feea143457c9e2c540e5c7c63b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.408553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c60feea143457c9e2c540e5c7c63b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.413376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:07.423557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c60feea143457c9e2c540e5c7c63b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.432812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c60feea143457c9e2c540e5c7c63b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.446794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2282] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:14:07 2022] DELETE /identity/v3/users/d2c60feea143457c9e2c540e5c7c63b7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:14:07.452812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:07.453313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:07.453415 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:07.471770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:14:07.473010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:07.515024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:07.534838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2283] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:14:07.544638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:07.546071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:07.605946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c727e-0c77-4d84-8673-14026b2ba3f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:07.611961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:07.613044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:07.613334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:07.613615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:07.625129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:07.626586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:07.637030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:14:07.915424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:07.917024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:07.933092 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c1bb-97c9-4aa4-a2d8-bffa01b050de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:07.938035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:07.938927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:07.939212 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:07.939477 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] PATH_INFO: `/v3/projects/4fc64c9b7cd84031b153d4548ea6d810` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:07.940941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc64c9b7cd84031b153d4548ea6d810)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:07.944315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:07.993325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc64c9b7cd84031b153d4548ea6d810', 'request_id': 'req-643958be-663b-4f87-8591-1287ca5bc54c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:07.999081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc64c9b7cd84031b153d4548ea6d810', 'request_id': 'req-643958be-663b-4f87-8591-1287ca5bc54c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:08.005388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc64c9b7cd84031b153d4548ea6d810', 'request_id': 'req-643958be-663b-4f87-8591-1287ca5bc54c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:08.020405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] The token cache is being invalidate because project 4fc64c9b7cd84031b153d4548ea6d810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:14:08.020729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:08.021373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-643958be-663b-4f87-8591-1287ca5bc54c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:14:08.022982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2285] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:14:07 2022] DELETE /identity/v3/projects/4fc64c9b7cd84031b153d4548ea6d810 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:14:17.632105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=36,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:18.017507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:25.963805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:25.973207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510cdd4f-b29a-4668-b2ea-e082b5186d1c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:26.025719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:26.026378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:26.026646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:26.027890 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:26.029848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:26.039850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:26.041445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:26.053184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:14:28.530175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:28.530726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:28.530726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:28.550728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] MFA Rules not processed for user `8bd2fc9f3216425cbcbbed84b7815529`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:14:28.551995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:28.611906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2287] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:14:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:14:33.813997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:33.816046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:33.900757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c4066-3c21-44d4-8e90-2648f6df6a92 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:33.907527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:33.908120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:33.908501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:33.908846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:33.911225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:33.923394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:33.925499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:33.936561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2288] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:14:38.596641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:38.613110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75995b8e-b50b-4a3a-8655-3a98fac0861e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:38.619338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:38.619987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:38.620274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:38.620555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:38.622752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:38.685161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:38.687530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:38.697332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:14:43.932463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:48.321170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:14:48.322739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:48.337274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e778001-f2e9-4650-b13a-1231077ae9c7 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:14:48.348381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:14:48.349084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:14:48.349373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:14:48.349652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:14:48.352148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:14:48.420355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:14:48.421918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f156-a554-4de3-afd7-3dc9a269914b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:14:48.431713 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:14:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:14:48.692999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=53 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:14:58.427326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:15:02.056331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:02.069526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336a2b6-4959-4844-a295-8854f3442b06 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:02.077078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:02.077758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:02.078051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:02.078336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:02.080433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:02.093816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:02.095926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:02.121684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2291] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:15:03.294820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9358bc-9baf-4194-b6b9-4924fa48c3d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:03.295466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9358bc-9baf-4194-b6b9-4924fa48c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:03.295466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9358bc-9baf-4194-b6b9-4924fa48c3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:03.313392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f9358bc-9baf-4194-b6b9-4924fa48c3d7 None None] MFA Rules not processed for user `41c4d3bab2164767b8eefcb4427b65b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:15:03.314683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f9358bc-9baf-4194-b6b9-4924fa48c3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:03.381418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2292] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:15:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:15:03.827697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:03.829204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:03.841774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe7bc-084a-47ba-8e84-f8a76cf1b993 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:03.845687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:03.846340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:03.846625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:03.846963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:03.848765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:03.859321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:03.860908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:03.869435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2293] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:15:04.805480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:04.806036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:04.806036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:04.827353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] MFA Rules not processed for user `5bb9100dcf2a4522840d726d800fe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:15:04.828608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:04.836758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2294] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:15:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:15:05.489911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:05.492303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:05.497788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:05.499440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:05.572396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5288813-0afd-4f5a-a085-1fb0c947a659 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:05.577662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:05.578323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:05.578607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:05.578885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:05.581009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:05.593751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:05.595692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:05.597264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0865cb0-cbae-4c35-aed2-2de3ab164ec6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:05.606278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2295] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:15:05.611760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:05.612521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:05.612815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:05.613096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:05.623104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:05.624800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:05.635159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2296] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:15:14.578317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:14.593671 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2276603-756e-4545-bbec-08f6670ec086 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:14.600583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:14.601483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:14.601775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:14.602057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:14.604071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:14.617859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:14.620138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a246090-9d82-40d1-b87f-0413bb36c004 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:14.632375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2297] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:15:15.630719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:15:20.703380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:20.729933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f89477-b193-4975-aadc-c4a8c3e932d3 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:20.739556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:20.740851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:20.741149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:20.741657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:20.743710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:20.755896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:20.758132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:20.769225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2298] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:15:21.923365 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:21.923895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:21.923993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:21.941905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] MFA Rules not processed for user `b778c9e1d53c44edb54b0c94c458e20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:15:21.943470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:22.002970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2299] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:15:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:15:22.319353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:22.321901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:22.340460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95f68c-ed4c-4f90-babd-98f6bc8e224b service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:22.346127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:22.347147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:22.347434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:22.347716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:22.349671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:22.360074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:22.362057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:22.370541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2300] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:15:31.995607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=75 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:15:32.367095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=12 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:15:59.235612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:15:59.251816 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff90a3-3f7b-452c-9b84-32564388657d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:15:59.259087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:15:59.260042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:15:59.260453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:15:59.260826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:15:59.264026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:15:59.331050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:15:59.332549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:15:59.342320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:02.484225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:02.498646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c91f0-85c7-4712-9f05-c08ea4952a8b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:02.504279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:02.505018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:02.505293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:02.505573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:02.507359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:02.520672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:02.522471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:02.537362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:02.550020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:02.551520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:02.567299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749b6d8-dbfe-4a2f-bddd-6b1b50b8eaed service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:02.572250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:02.573848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:02.574169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:02.574501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] PATH_INFO: `/v3/users/8bd2fc9f3216425cbcbbed84b7815529` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:02.577153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd2fc9f3216425cbcbbed84b7815529)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:02.581175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:02.631398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd2fc9f3216425cbcbbed84b7815529'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.632057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd2fc9f3216425cbcbbed84b7815529'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.643981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd2fc9f3216425cbcbbed84b7815529'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.644648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd2fc9f3216425cbcbbed84b7815529'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.651611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0221968b-3795-4e82-92f6-c398d0da4c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd2fc9f3216425cbcbbed84b7815529'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.666503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2303] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:02 2022] DELETE /identity/v3/users/8bd2fc9f3216425cbcbbed84b7815529 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:02.854986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:02.866321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bc1a9-8025-4ef2-8594-30a1f67d1b05 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:02.933348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:02.934225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:02.934537 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:02.934816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] PATH_INFO: `/v3/projects/7262f17284a44afaa2b5d2d2db0a3ef6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:02.936002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7262f17284a44afaa2b5d2d2db0a3ef6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:02.937958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:02.990799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7262f17284a44afaa2b5d2d2db0a3ef6', 'request_id': 'req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:02.996492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7262f17284a44afaa2b5d2d2db0a3ef6', 'request_id': 'req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:03.005686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7262f17284a44afaa2b5d2d2db0a3ef6', 'request_id': 'req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:03.017448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] The token cache is being invalidate because project 7262f17284a44afaa2b5d2d2db0a3ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:03.017743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:03.018485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06a41fcd-1003-438b-a0bf-c733bc8c2dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:03.020185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2304] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:02 2022] DELETE /identity/v3/projects/7262f17284a44afaa2b5d2d2db0a3ef6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:03.025445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.025735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.026006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.044211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:03.045462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:03.110141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2305] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:03.118078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.118368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.118657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.139777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:03.141051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:03.150935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2306] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:03.155579 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.170824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e836b0-95cd-460a-b6bb-c1ea11a321fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.176171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.176884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.177170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.177464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.178203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:16:03.178509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.180138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.182035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:03.196912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2307] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:16:03.201532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.211367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabdfb7-fcd0-488b-9658-46db4c3abdb5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.216437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.217133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.217417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.217694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.218548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.220517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.260006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2308] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:16:03.264107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.274547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418c6a3-1d10-41a6-8d52-80e1472614d6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.333044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.333740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.334031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.334315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.335032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.337000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.362928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2309] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:16:03.364103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.374898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89455197-3267-4357-ac4c-679f5cb8318d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.380253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.381053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.381361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.381602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.382323 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.383911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.398174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2310] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:03.399374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.409815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88f06-c23e-43b7-a3ae-8c7d852d465d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.415681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.416331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.416649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.416925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.417667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.419297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26685557-bca5-4053-ae24-dc0ff181a6b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.430438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2311] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:03.435385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.447202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f81e1-045d-40b0-baeb-2f08eaf261cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.455097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.456275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.456704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.457157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] PATH_INFO: `/v3/projects/11511495e80346448fb83a6682f92d8f/users/c66622e0d4d94206aebdf7181622fdb1/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.475027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11511495e80346448fb83a6682f92d8f, user_id=c66622e0d4d94206aebdf7181622fdb1, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.477611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.494087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2312] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:03 2022] PUT /identity/v3/projects/11511495e80346448fb83a6682f92d8f/users/c66622e0d4d94206aebdf7181622fdb1/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:03.504888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.505208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.505492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.533029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] MFA Rules not processed for user `c66622e0d4d94206aebdf7181622fdb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:03.534262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:03.601643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2313] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:03.606197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.611943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:03.680403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3592-6c94-4020-a119-5225caff621b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.685557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.686211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.686500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.686782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.687462 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.689616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.718670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2314] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:03.723138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.734183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d771e7-004e-46d8-b6c5-978a31e1e291 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.791947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.792651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.792940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.793216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.793897 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.795878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.803535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:03.818678 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2315] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:03 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:03.822656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.833362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1bca3-94b5-4c9c-9f3a-4cc02fdce9a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.838292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.838961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.839244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.839521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.840252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.841899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.852788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2316] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:03.857162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.868260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dd52a-2b96-4c88-9849-ebef9a302ae4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.874929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.875855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.876207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.876538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] PATH_INFO: `/v3/projects/7c9ca6361ef744eda242d6980157492a/users/4391252286864cd4bc727c1a10b561a1/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.894273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9ca6361ef744eda242d6980157492a, user_id=4391252286864cd4bc727c1a10b561a1, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:03.897038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:03.912271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2317] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:03 2022] PUT /identity/v3/projects/7c9ca6361ef744eda242d6980157492a/users/4391252286864cd4bc727c1a10b561a1/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:03.916440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:03.927433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d67d-8961-4731-aad3-4e6695fb0d65 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:03.996069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:03.996778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:03.997063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:03.997658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:03.998386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:04.000002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e108f04-e6fb-450d-a368-065168b1870f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:04.010134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2318] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:04.014632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:04.025750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173532c-e857-4595-a903-8fe857f31fdc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:04.031249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:04.032147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.032537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.032872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] PATH_INFO: `/v3/projects/7c9ca6361ef744eda242d6980157492a/users/4391252286864cd4bc727c1a10b561a1/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.035106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9ca6361ef744eda242d6980157492a, user_id=4391252286864cd4bc727c1a10b561a1, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:04.037884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:04.050819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2319] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:04 2022] PUT /identity/v3/projects/7c9ca6361ef744eda242d6980157492a/users/4391252286864cd4bc727c1a10b561a1/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:04.062940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.063263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.063719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.091374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] MFA Rules not processed for user `4391252286864cd4bc727c1a10b561a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:04.092630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:04.164548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:04.168707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2320] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:04.181804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:04.183600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:04.240752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58716275-425b-44af-bff9-beb7155d32c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:04.244877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:04.245526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.245806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.246078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.255599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:04.257082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc924d4-aa6d-4ebf-aaba-5ae6be4101fa service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:04.266583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2321] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:04.333077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:04.409400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be3aaf-b569-4336-bd7d-434b92aefec0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:04.414647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:04.415465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.415750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.416025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] PATH_INFO: `/v3/users/c66622e0d4d94206aebdf7181622fdb1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.417313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66622e0d4d94206aebdf7181622fdb1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:04.419250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:04.474635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66622e0d4d94206aebdf7181622fdb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:04.475529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66622e0d4d94206aebdf7181622fdb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:04.476458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66622e0d4d94206aebdf7181622fdb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:04.491774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66622e0d4d94206aebdf7181622fdb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:04.499592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66622e0d4d94206aebdf7181622fdb1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:04.512143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2322] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:04 2022] DELETE /identity/v3/users/c66622e0d4d94206aebdf7181622fdb1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:04.516946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.517240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.517513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.536523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:04.537764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:04.602643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2323] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:04.612526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:04.613969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:04.677520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ead2-3779-44fa-a1f5-238aa7861d41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:04.681436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:04.682093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:04.682380 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:04.682655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:04.692373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:04.693846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:04.703556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:04.990581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:04.992329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:05.005167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358682c-ef7f-461e-94b1-5007285ba53e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:05.010191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:05.011136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.011484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.011767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] PATH_INFO: `/v3/projects/11511495e80346448fb83a6682f92d8f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.013062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] RBAC: Authorizing `identity:delete_project(project_id=11511495e80346448fb83a6682f92d8f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:05.015474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:05.066699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11511495e80346448fb83a6682f92d8f', 'request_id': 'req-37574941-02b3-4612-8f2c-7e8845e5898b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.068757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:05.075868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11511495e80346448fb83a6682f92d8f', 'request_id': 'req-37574941-02b3-4612-8f2c-7e8845e5898b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.084066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11511495e80346448fb83a6682f92d8f', 'request_id': 'req-37574941-02b3-4612-8f2c-7e8845e5898b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.100002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] The token cache is being invalidate because project 11511495e80346448fb83a6682f92d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:05.100002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.100576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.102160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2325] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:04 2022] DELETE /identity/v3/projects/11511495e80346448fb83a6682f92d8f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:05.106209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:05.116817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5886-f1e1-4b20-b5b6-a280c4a46bfd service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:05.138351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:05.183115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:05.184282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.184605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.184883 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] PATH_INFO: `/v3/users/4391252286864cd4bc727c1a10b561a1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.186108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4391252286864cd4bc727c1a10b561a1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:05.188184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:05.236981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4391252286864cd4bc727c1a10b561a1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.237659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4391252286864cd4bc727c1a10b561a1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.238277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4391252286864cd4bc727c1a10b561a1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.250341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4391252286864cd4bc727c1a10b561a1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.258165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a425574-ddb7-406f-95ce-0a0325500179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4391252286864cd4bc727c1a10b561a1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.271015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2326] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:05 2022] DELETE /identity/v3/users/4391252286864cd4bc727c1a10b561a1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:05.554736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:05.556350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:05.571848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37574941-02b3-4612-8f2c-7e8845e5898b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:05.636372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:05.637294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.637568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.637846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] PATH_INFO: `/v3/projects/7c9ca6361ef744eda242d6980157492a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.639100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9ca6361ef744eda242d6980157492a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:05.641070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:05.693616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9ca6361ef744eda242d6980157492a', 'request_id': 'req-2b147066-2172-4721-ac17-6b723dc4399e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.699414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9ca6361ef744eda242d6980157492a', 'request_id': 'req-2b147066-2172-4721-ac17-6b723dc4399e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.706083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9ca6361ef744eda242d6980157492a', 'request_id': 'req-2b147066-2172-4721-ac17-6b723dc4399e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.721700 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] The token cache is being invalidate because project 7c9ca6361ef744eda242d6980157492a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:05.722005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.722607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b147066-2172-4721-ac17-6b723dc4399e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:05.724158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2327] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:05 2022] DELETE /identity/v3/projects/7c9ca6361ef744eda242d6980157492a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:05.730796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.731326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.731504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.750620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:05.752231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:05.827673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2328] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:05.835322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.835644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.835889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.854786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:05.856002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:05.865025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2329] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:05.869312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:05.883724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350071e-9edc-47e2-8ee5-2a56f5c9f144 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:05.889026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:05.889736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.890171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.890574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.891312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:16:05.891680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:05.893456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:05.918048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2330] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:16:05.922472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:05.932429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854826fc-7bc2-4e94-821d-4248060dee4d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:05.938569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:05.939459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:05.939731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:05.940007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:05.940775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:05.943534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:05.976478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2331] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:05 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:05.980836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:05.994864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aacbdb-f5f9-40e3-a99f-e194ddf868dd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.036225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:06.078856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.081018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.081312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.081597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.082324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.084545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.115997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2332] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:05 2022] POST /identity/v3/users => generated 302 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:06.120485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.135113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce7339-1d34-4e33-8412-3af4e4b9b48a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.141150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.142216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.142899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.143237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.143998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.146049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.174785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2333] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:16:06.178951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.190437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaad68-352e-4eda-918c-9ef996a3fd68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.195615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.196310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.196702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.196914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.197669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.199308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34764c25-75ef-4692-a87f-04c73a2b6acd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.211447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2334] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:06.215757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.228128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b0541-4fcb-43fa-9164-9f75f612a8ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.233089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.234067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.234387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.234672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] PATH_INFO: `/v3/projects/686cba40430c4780806be17175cd4d9c/users/7499db79fedd4cedaa0cb66fdb7b8da7/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.251769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=686cba40430c4780806be17175cd4d9c, user_id=7499db79fedd4cedaa0cb66fdb7b8da7, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.254554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.270421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2335] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:06 2022] PUT /identity/v3/projects/686cba40430c4780806be17175cd4d9c/users/7499db79fedd4cedaa0cb66fdb7b8da7/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:06.280708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.281057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.281418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.309860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] MFA Rules not processed for user `7499db79fedd4cedaa0cb66fdb7b8da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:06.311170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:06.376330 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2336] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:06.380863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.390648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:06.459972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09af7f0-bf09-444c-a5ce-86d4d05c91fc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.465107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.465787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.466062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.466349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.467047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.469042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.499164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2337] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:06 2022] POST /identity/v3/projects => generated 409 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:06.503584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.515145 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2fcc3-2883-44ca-b2fb-ee9bd9869a15 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.593031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.593721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.594589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.594589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.595055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.597419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.618396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2338] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:06 2022] POST /identity/v3/users => generated 311 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:16:06.622936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.633787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849751ea-6b45-4312-a214-eeb17154cf6a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.638544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.639222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.639506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.639790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.640560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.642257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.653872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2339] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:06.658389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.669014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20f1b7-83e6-4e92-b70c-10c7a0632c0e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.674526 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.675423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.675708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.675998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] PATH_INFO: `/v3/projects/27e0b07c77eb45aaba675bac58794b2f/users/b44d6bc136514208ae5c112f554ccb8f/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.692457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e0b07c77eb45aaba675bac58794b2f, user_id=b44d6bc136514208ae5c112f554ccb8f, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.695149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.710483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2340] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:06 2022] PUT /identity/v3/projects/27e0b07c77eb45aaba675bac58794b2f/users/b44d6bc136514208ae5c112f554ccb8f/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:06.714601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.725508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cc89d-3c2c-43f2-ba3d-cd30c6ec97b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.787348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.788044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.788311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.788617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.789356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.790994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f68c-464c-4915-b0b5-506fea227b93 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.805157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2341] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:06.806718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.814073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:06.821367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93384c8-a501-45d4-906d-53dfd6cc1c9a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:06.826439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:06.827336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.827619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.827895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] PATH_INFO: `/v3/projects/27e0b07c77eb45aaba675bac58794b2f/users/b44d6bc136514208ae5c112f554ccb8f/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.831058 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e0b07c77eb45aaba675bac58794b2f, user_id=b44d6bc136514208ae5c112f554ccb8f, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:06.833752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:06.845248 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2342] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:06 2022] PUT /identity/v3/projects/27e0b07c77eb45aaba675bac58794b2f/users/b44d6bc136514208ae5c112f554ccb8f/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:06.856766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:06.857008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:06.857320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:06.885161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] MFA Rules not processed for user `b44d6bc136514208ae5c112f554ccb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:06.886380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:06.977077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2343] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:06.989671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:06.991333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:07.052007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fc365-d69f-48a0-b4f8-030ffa347b26 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:07.055813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:07.056502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:07.056793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:07.057072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:07.066450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:07.067888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de8660-ed0d-4399-a30e-f587728e4d1e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:07.077703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:16:07.252882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:07.257762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:07.263499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca32af-0c59-4fe1-99c9-b60682fd6427 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:07.268769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:07.269444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:07.269924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:07.270200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:07.330653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:07.332292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:07.342225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:08.923157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15215-d302-486d-9f2f-6bc37534de04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:08.923671 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15215-d302-486d-9f2f-6bc37534de04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:08.923768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f15215-d302-486d-9f2f-6bc37534de04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:08.941682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28f15215-d302-486d-9f2f-6bc37534de04 None None] MFA Rules not processed for user `937fff3ec09e4f8a9a9a5fcdf2027a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:08.942955 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28f15215-d302-486d-9f2f-6bc37534de04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:09.000144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2346] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:16.785739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:16.847337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876a568-395c-4687-a25c-416a58f3410b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:16.852615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:16.853497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:16.853772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:16.854041 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] PATH_INFO: `/v3/users/7499db79fedd4cedaa0cb66fdb7b8da7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:16.855344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7499db79fedd4cedaa0cb66fdb7b8da7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:16.857319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:16.906885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7499db79fedd4cedaa0cb66fdb7b8da7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:16.907589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7499db79fedd4cedaa0cb66fdb7b8da7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:16.919722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7499db79fedd4cedaa0cb66fdb7b8da7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:16.920430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7499db79fedd4cedaa0cb66fdb7b8da7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:16.927441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7499db79fedd4cedaa0cb66fdb7b8da7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:16.944338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2347] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:16 2022] DELETE /identity/v3/users/7499db79fedd4cedaa0cb66fdb7b8da7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:16.949008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:16.949515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:16.949639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:16.968007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:16.969282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:17.038095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2348] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:17.051666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:17.054020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:17.080798 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:17.120124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e7b21-6196-47c9-bdc2-e6b6adbf61a8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:17.124486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:17.125177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:17.125466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:17.126003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:17.138012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:17.139472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:17.149044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2349] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:17.430465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:17.432029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:17.445831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d07bb4-dd3e-4de8-a166-1dbcc64924ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:17.451699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:17.452639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:17.452920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:17.453194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] PATH_INFO: `/v3/projects/686cba40430c4780806be17175cd4d9c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:17.455131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686cba40430c4780806be17175cd4d9c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:17.457307 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:17.498180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:17.512453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686cba40430c4780806be17175cd4d9c', 'request_id': 'req-3eefab00-2624-418c-9f52-4f216bfcc054'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.518296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686cba40430c4780806be17175cd4d9c', 'request_id': 'req-3eefab00-2624-418c-9f52-4f216bfcc054'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.526900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686cba40430c4780806be17175cd4d9c', 'request_id': 'req-3eefab00-2624-418c-9f52-4f216bfcc054'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.538820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] The token cache is being invalidate because project 686cba40430c4780806be17175cd4d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:17.539132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.539751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.541357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2350] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:17 2022] DELETE /identity/v3/projects/686cba40430c4780806be17175cd4d9c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:17.545490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:17.555344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a39d-22f5-495d-b66b-7ded8880b303 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:17.615162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:17.616058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:17.616359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:17.616692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] PATH_INFO: `/v3/users/b44d6bc136514208ae5c112f554ccb8f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:17.618037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44d6bc136514208ae5c112f554ccb8f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:17.620003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:17.668546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44d6bc136514208ae5c112f554ccb8f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.669249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44d6bc136514208ae5c112f554ccb8f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.681337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44d6bc136514208ae5c112f554ccb8f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.681937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44d6bc136514208ae5c112f554ccb8f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.689166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7294f403-7e41-4706-bd79-67a43a7dd796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44d6bc136514208ae5c112f554ccb8f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:17.705249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2351] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:17 2022] DELETE /identity/v3/users/b44d6bc136514208ae5c112f554ccb8f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:18.037818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.039327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:18.053823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefab00-2624-418c-9f52-4f216bfcc054 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.112151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.113079 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.113368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.113649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] PATH_INFO: `/v3/projects/27e0b07c77eb45aaba675bac58794b2f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.115029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e0b07c77eb45aaba675bac58794b2f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.117014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.170933 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e0b07c77eb45aaba675bac58794b2f', 'request_id': 'req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:18.177387 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e0b07c77eb45aaba675bac58794b2f', 'request_id': 'req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:18.186396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e0b07c77eb45aaba675bac58794b2f', 'request_id': 'req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:18.198303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] The token cache is being invalidate because project 27e0b07c77eb45aaba675bac58794b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:18.198535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:18.199191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a36a227-7afa-4c89-ba8f-eed15ee31c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:18.200869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2352] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:18 2022] DELETE /identity/v3/projects/27e0b07c77eb45aaba675bac58794b2f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:18.206210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.206693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.206797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.226241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:18.227328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:18.267341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:18.289879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2353] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:18.297606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.297902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.298188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.318622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:18.319849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:18.328820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2354] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:18.333363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.343217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de69a42-dc43-4e9d-85fa-b4b9fb2190bf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.348153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.348857 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.349149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.349430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.350203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:16:18.350519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.352186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.364624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2355] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:16:18.370807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.381781 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f51b38-14c4-437e-8cfe-0eaca8059789 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.388034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.388776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.389054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.389325 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.390027 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.391988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.419865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2356] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:18.424047 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.434467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f83ac-80dc-4bc4-a459-60d36b0dd4ae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.492804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.493528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.493818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.494098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.494789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.496740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.522116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2357] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:18.523582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.534494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2159a1-26d7-4bd9-99da-51c29b03318c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.540038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.545803 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.546576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.546929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.547790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.549455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.565744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2358] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:18.567023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.577346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6df27-4ba4-4104-8a5b-e338a0693a21 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.582839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.583509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.583792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.584070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.584869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.586485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b986af-1e10-46e8-9254-0e82c5ca50a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.597026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2359] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:18.601255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.611447 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182502c1-c39f-4230-8ca9-24e811a7fa36 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.616392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.617324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.617613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.617893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] PATH_INFO: `/v3/projects/808c05db5d344112bb215739acedbc90/users/70d54627b3f44ae9bdd0e24a169b584b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.626861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:18.636015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808c05db5d344112bb215739acedbc90, user_id=70d54627b3f44ae9bdd0e24a169b584b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.639068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.653926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2360] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:18 2022] PUT /identity/v3/projects/808c05db5d344112bb215739acedbc90/users/70d54627b3f44ae9bdd0e24a169b584b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:18.665727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.665983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.666288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.694079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] MFA Rules not processed for user `70d54627b3f44ae9bdd0e24a169b584b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:18.695343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:18.762463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2361] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:18.767054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.829547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075c5c6-1605-48e8-9faf-7f313fae2cc4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.834449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.835121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.835404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.835682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.836375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.838326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.865734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2362] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:18.870325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.881964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53d2fd-90d7-418c-abe1-af303d19e444 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.914313 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:18.941009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.941682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.941966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.942247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.942945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.944915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:18.968472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2363] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:18 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:18.969639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:18.980274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e4e36-935b-4e3d-81b1-1fac4a12e3bd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:18.984952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:18.985678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:18.985966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:18.986242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:18.986997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:18.988658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:19.001131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2364] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:19.005429 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:19.017768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae30a-51fb-4eb3-bcc5-4ff1931e0c0d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:19.022684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:19.023611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:19.023892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:19.024167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] PATH_INFO: `/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5/users/111cbbaf15a04547825535d152a058a6/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:19.040252 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9ed8465a9646a5a410c3aaf8058ad5, user_id=111cbbaf15a04547825535d152a058a6, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:19.043371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:19.057201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2365] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:19 2022] PUT /identity/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5/users/111cbbaf15a04547825535d152a058a6/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:19.061367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:19.072059 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361025-2086-4237-85b6-10154ef1a48f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:19.134385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:19.135057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:19.135341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:19.135617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:19.136350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:19.137971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043611c-699e-43e7-82e0-e8aa7c809f11 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:19.148350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2366] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:19.152762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:19.163406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b62b7-1a52-4fbd-adc9-e955da31400f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:19.168159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:19.169082 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:19.169361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:19.169635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] PATH_INFO: `/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5/users/111cbbaf15a04547825535d152a058a6/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:19.171598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9ed8465a9646a5a410c3aaf8058ad5, user_id=111cbbaf15a04547825535d152a058a6, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:19.174561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:19.185520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2367] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:19 2022] PUT /identity/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5/users/111cbbaf15a04547825535d152a058a6/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:19.195899 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:19.196216 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:19.196534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:19.230745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] MFA Rules not processed for user `111cbbaf15a04547825535d152a058a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:19.231936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:19.279935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:19.307679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2368] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:19.322711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:19.324658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:19.387641 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab789987-cedc-4241-941f-c065a7e75596 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:19.392009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:19.392736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:19.393023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:19.393304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:19.450401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:19.451855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:19.460969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:23.715244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:23.716769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:23.729427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92b97b-1cd3-4c95-a3da-3cd1ebfb7c48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:23.733628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:23.734314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:23.734569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:23.734851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:23.744296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:23.745763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b755b-a8a0-46eb-a2a6-d1ccae47bcb5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:23.756225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2370] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:28.651868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:28.706627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:28.725369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a3c4b-b98f-4f00-819e-9188678098b2 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:28.730512 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:28.731394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:28.731683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:28.731963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] PATH_INFO: `/v3/users/70d54627b3f44ae9bdd0e24a169b584b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:28.733247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d54627b3f44ae9bdd0e24a169b584b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:28.735799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:28.790398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d54627b3f44ae9bdd0e24a169b584b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:28.791073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d54627b3f44ae9bdd0e24a169b584b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:28.802839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d54627b3f44ae9bdd0e24a169b584b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:28.803484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d54627b3f44ae9bdd0e24a169b584b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:28.810757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d54627b3f44ae9bdd0e24a169b584b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:28.826476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2371] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:28 2022] DELETE /identity/v3/users/70d54627b3f44ae9bdd0e24a169b584b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:28.831159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ce25-1790-4600-8738-e351156034d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:28.831638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ce25-1790-4600-8738-e351156034d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:28.831733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521ce25-1790-4600-8738-e351156034d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:28.849850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9521ce25-1790-4600-8738-e351156034d5 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:28.851207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9521ce25-1790-4600-8738-e351156034d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:28.925024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2372] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:28.934907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:28.936635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:29.003690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ab53f-08b4-4c79-8d26-ce78f6325939 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:29.008336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:29.009020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:29.009307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:29.009586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:29.019727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:29.021211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:29.031208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2373] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:29.305519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521ce25-1790-4600-8738-e351156034d5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:29.306958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9521ce25-1790-4600-8738-e351156034d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:29.319043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521ce25-1790-4600-8738-e351156034d5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:29.323953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:29.324989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:29.325250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:29.325528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] PATH_INFO: `/v3/projects/808c05db5d344112bb215739acedbc90` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:29.326751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=808c05db5d344112bb215739acedbc90)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:29.328764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:29.378055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808c05db5d344112bb215739acedbc90', 'request_id': 'req-2e3de797-a5d6-45b5-971a-e7b9eaff7013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.383602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808c05db5d344112bb215739acedbc90', 'request_id': 'req-2e3de797-a5d6-45b5-971a-e7b9eaff7013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.392151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808c05db5d344112bb215739acedbc90', 'request_id': 'req-2e3de797-a5d6-45b5-971a-e7b9eaff7013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.403335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] The token cache is being invalidate because project 808c05db5d344112bb215739acedbc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:29.403629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.404249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.405837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2374] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:29 2022] DELETE /identity/v3/projects/808c05db5d344112bb215739acedbc90 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:16:29.410235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:29.420241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd385bf-e971-4d57-87a3-c3c08ae22e2e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:29.480255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:29.481150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:29.481436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:29.481708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] PATH_INFO: `/v3/users/111cbbaf15a04547825535d152a058a6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:29.482936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111cbbaf15a04547825535d152a058a6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:29.485241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:29.490751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:29.542305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111cbbaf15a04547825535d152a058a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.542991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111cbbaf15a04547825535d152a058a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.555710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111cbbaf15a04547825535d152a058a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.556382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111cbbaf15a04547825535d152a058a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.563576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-191e5631-340b-4318-abab-d232745956c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111cbbaf15a04547825535d152a058a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.578193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2375] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:29 2022] DELETE /identity/v3/users/111cbbaf15a04547825535d152a058a6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:29.861024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:29.862517 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:29.881606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de797-a5d6-45b5-971a-e7b9eaff7013 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:29.930132 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:29.940784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:29.941844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:29.942321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:29.942613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] PATH_INFO: `/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:29.944091 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9ed8465a9646a5a410c3aaf8058ad5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:29.946096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:29.992406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9ed8465a9646a5a410c3aaf8058ad5', 'request_id': 'req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:29.997957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9ed8465a9646a5a410c3aaf8058ad5', 'request_id': 'req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:30.006190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9ed8465a9646a5a410c3aaf8058ad5', 'request_id': 'req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:30.017372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] The token cache is being invalidate because project 8f9ed8465a9646a5a410c3aaf8058ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:30.017665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:30.018305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ff4e6c3-3d98-4e81-8e62-96526ce1e4c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:30.019874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2376] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:29 2022] DELETE /identity/v3/projects/8f9ed8465a9646a5a410c3aaf8058ad5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:30.026015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.026493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.026610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.044393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:30.045644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:30.108201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2377] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:30.117170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.117463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.117747 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.137186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:30.138407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:30.148012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2378] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:30.152382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.162998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c718fe-5b02-4681-86dc-3c5bbbcc5fac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.169021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.169697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.169966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.170243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.171014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:16:30.171337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.173764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.186509 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2379] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:16:30.191359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.201459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc797-334a-42df-9528-f9d092c982b3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.206560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.207294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.207648 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.207995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.208731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.210642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.239799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2380] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:30.244132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.255361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade8f8-34aa-4d98-a632-6d0c8e540c1a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.316197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.316925 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.317227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.317494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.318184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.320162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.343221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2381] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:30.347496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.358172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8a930-29ba-4075-9d45-742a888c8513 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.363131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.363793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.364076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.364350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.365109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.366708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.377969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2382] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:30.381735 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.386797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:30.392626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8cc25-3490-493f-813d-b816c2c6d810 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.398180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.398886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.399179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.399615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.400363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.402381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57637013-44ff-424d-81d1-74e63541c195 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.413542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2383] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:30.417711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.428192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11c95f-8ca9-49f7-a240-9c50b5619710 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.433706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.434596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.434886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.435163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] PATH_INFO: `/v3/projects/698679afc8b74e0e99f29c12e49a08ac/users/7b83b34403564c6e8e00c5a7ba790cea/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.450837 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=698679afc8b74e0e99f29c12e49a08ac, user_id=7b83b34403564c6e8e00c5a7ba790cea, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.453825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.468161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2384] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:30 2022] PUT /identity/v3/projects/698679afc8b74e0e99f29c12e49a08ac/users/7b83b34403564c6e8e00c5a7ba790cea/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:30.480602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.480898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.481178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.508652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] MFA Rules not processed for user `7b83b34403564c6e8e00c5a7ba790cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:30.509772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:30.583331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2385] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:30.587898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.652361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cf322-671a-40f2-92f4-8b649197bff0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.657556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.658201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.658480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.658760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.659452 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.661381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.696209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2386] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:16:30.700449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.711821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbfedf-dfce-4f33-a9c7-1252745322e3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.769689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.770256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.770538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.770810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.771507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.773480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.794525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2387] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:30.798699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.805362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:30.809520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2046-93fb-4e19-b53d-73e580b7920f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.815605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.816312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.816634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.816920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.817659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.819760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.830707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2388] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:16:30.834742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.844769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951123aa-0755-407e-a651-53ce564d84b4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.849830 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.850488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.850758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.851042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.851756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.853383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198fb1-0be1-4769-bf23-874eacb39ada admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.864095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2389] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:30.868345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:30.879901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3cca9-b106-420d-b27b-8454d067b06b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:30.885714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:30.886600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.886885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.887163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] PATH_INFO: `/v3/projects/e0b32334c65245e0aa2a55b5e53f44d5/users/00ea3fd39f734561991d2cf5ba016116/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.902923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b32334c65245e0aa2a55b5e53f44d5, user_id=00ea3fd39f734561991d2cf5ba016116, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:30.905638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:30.921474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2390] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:30 2022] PUT /identity/v3/projects/e0b32334c65245e0aa2a55b5e53f44d5/users/00ea3fd39f734561991d2cf5ba016116/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:16:30.931890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:30.932185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:30.932500 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:30.961143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] MFA Rules not processed for user `00ea3fd39f734561991d2cf5ba016116`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:30.962364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:31.014005 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:31.033067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2391] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:31.037697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.102975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a221f2c-eefb-4333-a8a3-df9af6623215 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.108172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.108878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.109160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.109434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.110141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.112096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.138738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2392] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:31 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:31.143003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.154568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7591ef-afd6-4c4d-b3a7-1325771a76f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.212980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.213647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.213933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.214206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.214917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.216900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.243907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2393] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:31 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:16:31.248368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.259618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ee31-63e3-4f9b-9dec-ee86523845f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.265007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.265683 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.265989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.266409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.267265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.268919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.279916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2394] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:31.284603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.294882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6cbc1-a9ec-4825-b831-20d002d4a218 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.300088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.301069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.301353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.301628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] PATH_INFO: `/v3/projects/489f76c74eee4f47bc062108af61023e/users/382a661ab06c47ac9e35e708d922beb4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.316665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489f76c74eee4f47bc062108af61023e, user_id=382a661ab06c47ac9e35e708d922beb4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.319314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.334246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2395] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:31 2022] PUT /identity/v3/projects/489f76c74eee4f47bc062108af61023e/users/382a661ab06c47ac9e35e708d922beb4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:16:31.338308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.348522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a61cb-d93d-43cc-89c3-b4a4d3433268 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.410040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.410719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.411009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.411290 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.412019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.413666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf47d5-1f3d-4e33-baa6-cb2ecaa11529 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.424835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2396] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:16:31.429242 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.445129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfe9b-226d-4835-92c1-b825b29964f9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.454681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.455578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.455874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.456145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] PATH_INFO: `/v3/projects/489f76c74eee4f47bc062108af61023e/users/382a661ab06c47ac9e35e708d922beb4/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.458498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489f76c74eee4f47bc062108af61023e, user_id=382a661ab06c47ac9e35e708d922beb4, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.461215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.473699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2397] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:31 2022] PUT /identity/v3/projects/489f76c74eee4f47bc062108af61023e/users/382a661ab06c47ac9e35e708d922beb4/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:31.483962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.484253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.484573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.514072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] MFA Rules not processed for user `382a661ab06c47ac9e35e708d922beb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:31.515286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:31.546026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:31.589243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2398] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:31.602480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:31.604383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:31.663330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e462a-9a58-4d98-94d3-0799944f2ef5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:31.667474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:31.668125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:31.668443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:31.668720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:31.723730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:31.726159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:31.727912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:31.737361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2399] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:33.255014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:33.256694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:33.270218 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e573e8-5a6c-4d34-9da8-09c8ae9f38d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:33.274937 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:33.275586 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:33.275998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:33.276278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:33.343855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:33.345594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:33.355039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:33.912948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:33.922210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290da25-d106-461e-b837-2fd22cf3ed7f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:33.926561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:33.927197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:33.927479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:33.927753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:33.937331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:33.939010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac841-da8c-4af7-b807-eff5b890a2e6 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:33.948348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:37.833428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:37.899214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965d64-e0a6-4524-8150-7e8829a635d4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:37.904679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:37.905564 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:37.905840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:37.906150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] PATH_INFO: `/v3/users/7b83b34403564c6e8e00c5a7ba790cea` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:37.907625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b83b34403564c6e8e00c5a7ba790cea)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:37.909641 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:37.970547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b83b34403564c6e8e00c5a7ba790cea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:37.971195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b83b34403564c6e8e00c5a7ba790cea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:37.971956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b83b34403564c6e8e00c5a7ba790cea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:37.983895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b83b34403564c6e8e00c5a7ba790cea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:37.985647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:37.993253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b83b34403564c6e8e00c5a7ba790cea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.006270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2402] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:37 2022] DELETE /identity/v3/users/7b83b34403564c6e8e00c5a7ba790cea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:38.010937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:38.011407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:38.011515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:38.029900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:38.031143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:38.094155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2403] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:38.104038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:38.105579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:38.168041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db7fcb-2846-41a0-ab45-5dd68d4d4681 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:38.172172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:38.172935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:38.173280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:38.173483 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:38.183148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:38.184651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:38.194384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2404] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:38.481019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:38.482541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:38.496647 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850216-de72-4481-8ece-5dee22249d3a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:38.502844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:38.503731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:38.504063 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:38.504275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] PATH_INFO: `/v3/projects/698679afc8b74e0e99f29c12e49a08ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:38.505716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] RBAC: Authorizing `identity:delete_project(project_id=698679afc8b74e0e99f29c12e49a08ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:38.507628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:38.562385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698679afc8b74e0e99f29c12e49a08ac', 'request_id': 'req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.568207 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698679afc8b74e0e99f29c12e49a08ac', 'request_id': 'req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.574469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698679afc8b74e0e99f29c12e49a08ac', 'request_id': 'req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.589024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] The token cache is being invalidate because project 698679afc8b74e0e99f29c12e49a08ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:38.589332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.589964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.591563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2405] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:38 2022] DELETE /identity/v3/projects/698679afc8b74e0e99f29c12e49a08ac => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:38.596168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:38.605863 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155d42-6e38-48cc-aa6d-457cabab6dd5 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:38.676400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:38.677312 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:38.677598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:38.677876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] PATH_INFO: `/v3/users/00ea3fd39f734561991d2cf5ba016116` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:38.679144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ea3fd39f734561991d2cf5ba016116)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:38.681178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:38.732450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ea3fd39f734561991d2cf5ba016116'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.733210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ea3fd39f734561991d2cf5ba016116'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.733930 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ea3fd39f734561991d2cf5ba016116'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.747787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ea3fd39f734561991d2cf5ba016116'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.757463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ea3fd39f734561991d2cf5ba016116'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:38.770372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2406] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:38 2022] DELETE /identity/v3/users/00ea3fd39f734561991d2cf5ba016116 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:39.047231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:39.049489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:39.065958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4a71-1c9a-4be6-b613-255c7a87dcff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:39.129821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:39.130616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.130904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.131178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] PATH_INFO: `/v3/projects/e0b32334c65245e0aa2a55b5e53f44d5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.132373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b32334c65245e0aa2a55b5e53f44d5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:39.134372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:39.142730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:39.189403 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b32334c65245e0aa2a55b5e53f44d5', 'request_id': 'req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.195094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b32334c65245e0aa2a55b5e53f44d5', 'request_id': 'req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.201558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b32334c65245e0aa2a55b5e53f44d5', 'request_id': 'req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.216069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] The token cache is being invalidate because project e0b32334c65245e0aa2a55b5e53f44d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:39.216502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.217174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.218838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2407] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:39 2022] DELETE /identity/v3/projects/e0b32334c65245e0aa2a55b5e53f44d5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:39.223456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:39.224915 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:39.235096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:39.239410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0309c-6dc0-4c75-9724-20cbe90aad3f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:39.302222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:39.303113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.303399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.303680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] PATH_INFO: `/v3/users/382a661ab06c47ac9e35e708d922beb4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.304974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382a661ab06c47ac9e35e708d922beb4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:39.306903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:39.355902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382a661ab06c47ac9e35e708d922beb4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.356611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382a661ab06c47ac9e35e708d922beb4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.357263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382a661ab06c47ac9e35e708d922beb4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.369182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382a661ab06c47ac9e35e708d922beb4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.378828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1f34b08e-625b-4af2-8261-0d1e02f65b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382a661ab06c47ac9e35e708d922beb4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.390925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2408] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:16:39 2022] DELETE /identity/v3/users/382a661ab06c47ac9e35e708d922beb4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:39.668266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:39.669706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:39.683717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3b6d-ff1e-4c23-93af-3b6bb683c4bd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:39.742106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:39.743102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.743390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.743674 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] PATH_INFO: `/v3/projects/489f76c74eee4f47bc062108af61023e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.744987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=489f76c74eee4f47bc062108af61023e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:39.746924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:39.798018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489f76c74eee4f47bc062108af61023e', 'request_id': 'req-11c0addd-e188-43fb-b966-d4224a8ba13b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.804273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489f76c74eee4f47bc062108af61023e', 'request_id': 'req-11c0addd-e188-43fb-b966-d4224a8ba13b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.810858 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489f76c74eee4f47bc062108af61023e', 'request_id': 'req-11c0addd-e188-43fb-b966-d4224a8ba13b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.825802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] The token cache is being invalidate because project 489f76c74eee4f47bc062108af61023e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:16:39.826085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.826704 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-11c0addd-e188-43fb-b966-d4224a8ba13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:16:39.828273 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2409] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:16:39 2022] DELETE /identity/v3/projects/489f76c74eee4f47bc062108af61023e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:39.833765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.834084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.834371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.853105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:39.854334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:39.914547 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2410] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:39.922408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.923108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.923108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.942277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:39.943524 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:39.952847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2411] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:39.957408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:39.967615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3b33a-b908-4473-9e87-e9d52fad240f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:39.973990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:39.974650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:39.974906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:39.975237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:39.975972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:16:39.976282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:39.977909 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:39.989827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2412] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:16:39.994197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.004090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2a37-eaaa-4951-ab8b-05f0e6d101a1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.009878 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.010520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.010811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.011080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.011765 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.013703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.038216 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:40.050217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2413] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:16:39 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:40.055608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.066035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92f34-f566-4803-af4b-483f57c11c5a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.074603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:40.127857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.128580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.128867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.129142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.129876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.131802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.154858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2414] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:16:40 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:16:40.158934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.169821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87aedc-9df3-44b1-89d1-0800b0a6773d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.174947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.175614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.175896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.176170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.176944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.178561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.189440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2415] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:16:40.193572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.204511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e590e-e4e8-4e87-b4e4-7913e4cee7c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.211057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.211732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.212016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.212293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.213126 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.214680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbc5d3-cd39-4db8-ace8-539355932d6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.225599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2416] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:16:40.229601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.239881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869680b0-579d-4e02-8431-d296ad89adba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.245706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.246606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.246882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.247155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] PATH_INFO: `/v3/projects/40661b95bfb54299a52d803f7997d8f1/users/04c3c140de9d4f208e371999ae70f11e/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.263709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40661b95bfb54299a52d803f7997d8f1, user_id=04c3c140de9d4f208e371999ae70f11e, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.266749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.281369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2417] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:16:40 2022] PUT /identity/v3/projects/40661b95bfb54299a52d803f7997d8f1/users/04c3c140de9d4f208e371999ae70f11e/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:16:40.286980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fb99-8c73-4477-bb30-0b5d84dd74f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.287277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fb99-8c73-4477-bb30-0b5d84dd74f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.287554 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3fb99-8c73-4477-bb30-0b5d84dd74f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.305785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2bb3fb99-8c73-4477-bb30-0b5d84dd74f0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:40.306988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bb3fb99-8c73-4477-bb30-0b5d84dd74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:40.365796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2418] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:40.376490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:40.378373 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:40.438819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e014a58-7394-447d-bcea-a2173dc7ba30 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:40.443430 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:40.444087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:40.444484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:40.444799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:40.455928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:40.457439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:40.467203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:41.083559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656bccb-fd17-414c-ab97-f1882fc3f7df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:41.083855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656bccb-fd17-414c-ab97-f1882fc3f7df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:41.084141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656bccb-fd17-414c-ab97-f1882fc3f7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:41.112314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1656bccb-fd17-414c-ab97-f1882fc3f7df None None] MFA Rules not processed for user `04c3c140de9d4f208e371999ae70f11e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:41.113575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1656bccb-fd17-414c-ab97-f1882fc3f7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:41.185584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2420] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:41.196562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:16:41.198232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:41.259294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495046b-5318-49ab-84a2-a3db7fabe153 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:16:41.263088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:16:41.263762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:41.264045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:41.264323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:41.273880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:16:41.275326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:16:41.284112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:16:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:16:42.658891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135bb33-04d3-4552-9e3c-b9ad9044b267 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:16:42.659488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135bb33-04d3-4552-9e3c-b9ad9044b267 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:16:42.659488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135bb33-04d3-4552-9e3c-b9ad9044b267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:16:42.681861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5135bb33-04d3-4552-9e3c-b9ad9044b267 None None] MFA Rules not processed for user `04c3c140de9d4f208e371999ae70f11e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:16:42.683076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5135bb33-04d3-4552-9e3c-b9ad9044b267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:16:42.691241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2422] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:16:51.280226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:16:52.676225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:00.053528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:00.054986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:00.130016 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000f6b0-f29c-4430-8fb5-6db58fdfceb3 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:00.137885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:00.138889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:00.139286 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:00.139692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:00.142961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:00.155649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:00.157278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00ec9f-6679-4588-8515-c8bdd1917da3 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:00.167756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2423] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:17:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:17:01.590480 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d8f7b-d7d6-4260-9d23-ffb9b57e8c53 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:01.590989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d8f7b-d7d6-4260-9d23-ffb9b57e8c53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:01.590989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d8f7b-d7d6-4260-9d23-ffb9b57e8c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:01.608893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-496d8f7b-d7d6-4260-9d23-ffb9b57e8c53 None None] MFA Rules not processed for user `04c3c140de9d4f208e371999ae70f11e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:01.610164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-496d8f7b-d7d6-4260-9d23-ffb9b57e8c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:01.617675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2424] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:08.973473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:08.974013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:08.974013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:08.991645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] MFA Rules not processed for user `5bb9100dcf2a4522840d726d800fe013`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:08.992903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:09.054206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2425] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:11.603548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:19.046534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=57 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:31.694404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d8c79-ba1e-41c3-b97f-1f8b57586669 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:31.694946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d8c79-ba1e-41c3-b97f-1f8b57586669 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:31.694946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d8c79-ba1e-41c3-b97f-1f8b57586669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:31.714196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb5d8c79-ba1e-41c3-b97f-1f8b57586669 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:31.715398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb5d8c79-ba1e-41c3-b97f-1f8b57586669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:31.777701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2426] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:17:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:31.845945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:31.847774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:31.923299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83ea0e-b3a3-4e9b-923a-907716946d93 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:31.928682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:31.929340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:31.929591 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:31.929905 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:31.938266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:31.939867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:31.948122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2427] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:17:32.474457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2b54-7944-4db1-aeb5-d8ad8e207ed8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:32.474763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2b54-7944-4db1-aeb5-d8ad8e207ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:32.475445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2b54-7944-4db1-aeb5-d8ad8e207ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:32.496255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-afde2b54-7944-4db1-aeb5-d8ad8e207ed8 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:32.497941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afde2b54-7944-4db1-aeb5-d8ad8e207ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:32.511226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2428] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:17:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:32.521235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:32.537077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456ed66-3cd9-48ac-8d29-427166b099c8 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:32.542217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:32.542873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:32.543168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:32.543421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:32.560008 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:32.561673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:32.577681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2429] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:17:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:17:42.489140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:42.569692 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=38 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:49.208808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8655-450f-49fc-bcae-77f8c4d45578 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:49.209388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8655-450f-49fc-bcae-77f8c4d45578 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:49.209388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e8655-450f-49fc-bcae-77f8c4d45578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:49.229405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-678e8655-450f-49fc-bcae-77f8c4d45578 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:49.230636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-678e8655-450f-49fc-bcae-77f8c4d45578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:49.248746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2430] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:17:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:49.258821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:49.270727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5bca-1f2c-4353-b5df-ca2734c7294a service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:49.276670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:49.277705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:49.278029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:49.278371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:49.288487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:49.290124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:49.300129 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2431] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:17:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:17:53.220563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:53.221088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:53.221194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:53.239310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:53.240560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:53.248702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2432] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:17:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:53.261256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:53.272520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc700e-8bbe-491c-a879-7b3844496e3b service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:53.277854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:53.278607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:53.278982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:53.279323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:53.288434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:53.290134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:53.298659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2433] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:17:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:17:54.542367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:54.557850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594cb90-ac04-406b-b681-a364d3d401ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:54.562388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:54.563342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:54.563712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:54.564099 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] PATH_INFO: `/v3/users/04c3c140de9d4f208e371999ae70f11e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:54.565406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c3c140de9d4f208e371999ae70f11e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:54.567336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:54.619983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c3c140de9d4f208e371999ae70f11e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.620639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c3c140de9d4f208e371999ae70f11e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.621252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c3c140de9d4f208e371999ae70f11e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.633397 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c3c140de9d4f208e371999ae70f11e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.641573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9e2852-7515-4d35-a098-20225726b11e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c3c140de9d4f208e371999ae70f11e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.655012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2434] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:17:54 2022] DELETE /identity/v3/users/04c3c140de9d4f208e371999ae70f11e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:17:54.843896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:54.855020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eceace-8a16-4fa0-8b45-8cf34f9e1841 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:54.928138 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:54.929049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:54.929335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:54.929610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] PATH_INFO: `/v3/projects/40661b95bfb54299a52d803f7997d8f1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:54.930811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40661b95bfb54299a52d803f7997d8f1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:54.932801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:54.982767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40661b95bfb54299a52d803f7997d8f1', 'request_id': 'req-c9300aaa-067d-4585-988e-1429e5412ed1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.988208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40661b95bfb54299a52d803f7997d8f1', 'request_id': 'req-c9300aaa-067d-4585-988e-1429e5412ed1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:54.994593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40661b95bfb54299a52d803f7997d8f1', 'request_id': 'req-c9300aaa-067d-4585-988e-1429e5412ed1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:55.008784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] The token cache is being invalidate because project 40661b95bfb54299a52d803f7997d8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:17:55.009073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:55.009739 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c9300aaa-067d-4585-988e-1429e5412ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:17:55.015174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2435] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:17:54 2022] DELETE /identity/v3/projects/40661b95bfb54299a52d803f7997d8f1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:17:55.018413 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.018711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.019000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.040063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:55.041598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:55.110279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2436] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:55.118468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.118760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.119043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.138035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:55.139169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:55.147750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2437] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:55.152693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.163995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbbec1-5d54-48dc-b5e0-f1976bd92f3c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.169895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.170983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.171375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.171626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.172405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:17:55.172750 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.174460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.186997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2438] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:17:55.191722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.201134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd4b9e-5d22-4609-8155-51ac2fd49b38 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.206498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.207155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.207439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.207720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.208451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.210399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.240652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2439] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:17:55 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:17:55.245108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.255625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b4cb5-e9ab-4a4b-a7b3-921ee10ac7f8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.320186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:55.336719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.337395 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.337684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.337962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.338659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.340606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.375439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2440] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:17:55 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:17:55.379810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.407991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463aebf-779f-49e2-bde4-e98d8086fa1c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.413654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.414462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.414845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.415222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.416081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.417846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.431361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2441] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:17:55.435474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.448352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6b7ad-bb20-4eb8-bf65-e8cd656a2b87 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.453805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.454941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.455475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.455828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.456767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.458453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6778d8d-1ff4-41df-835c-c9042d759814 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.472487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2442] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:17:55.474203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.484274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80717fd2-f0f9-44b5-8780-e142b70be6a1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.489523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.490654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.490940 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.491353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] PATH_INFO: `/v3/projects/333e2220adfb4ed58ffceb41915e8d90/users/cca17596a22a49089b6f42ab8b96a543/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.510584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=333e2220adfb4ed58ffceb41915e8d90, user_id=cca17596a22a49089b6f42ab8b96a543, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.513712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.525567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:17:55.531026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2443] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:17:55 2022] PUT /identity/v3/projects/333e2220adfb4ed58ffceb41915e8d90/users/cca17596a22a49089b6f42ab8b96a543/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:17:55.596527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8709-7df6-458e-85db-728d955d8048 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.597153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8709-7df6-458e-85db-728d955d8048 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.597153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8709-7df6-458e-85db-728d955d8048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.624918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c64c8709-7df6-458e-85db-728d955d8048 None None] MFA Rules not processed for user `cca17596a22a49089b6f42ab8b96a543`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:55.626133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c64c8709-7df6-458e-85db-728d955d8048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:55.694790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2444] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:17:55.706943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:17:55.709531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:55.778757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92b96d-1c50-4646-8690-b8a68e79bb74 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:17:55.782428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:17:55.783268 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:55.783552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:55.783829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:55.792693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:17:55.794158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79882231-361d-4ef2-8da2-dfb91b998e16 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:17:55.804024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:17:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:17:59.778928 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:17:59.779672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:17:59.779672 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:17:59.803973 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] MFA Rules not processed for user `41c4d3bab2164767b8eefcb4427b65b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:17:59.805485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:17:59.886422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2446] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:17:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:05.799775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:09.879697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:21.264196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:21.264681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:21.264778 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:21.282495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:21.283698 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:21.344454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2447] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:18:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:21.357197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:21.358636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:21.432063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636b240-465e-4e37-82bb-47a742ebc24d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:21.437782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:21.438550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:21.438836 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:21.439112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:21.448384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:21.450049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ef176-886e-43a7-95b4-27005d6f911a service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:21.459688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2448] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:18:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:18:31.337592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:31.456831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=32 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:37.401548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:37.468073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7561c1-fabb-472c-8e41-bf438d3b5bc7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:37.475730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:37.476688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:37.477122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:37.477419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] PATH_INFO: `/v3/users/cca17596a22a49089b6f42ab8b96a543` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:37.478764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca17596a22a49089b6f42ab8b96a543)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:37.480834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:37.536343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca17596a22a49089b6f42ab8b96a543'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:37.537114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca17596a22a49089b6f42ab8b96a543'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:37.552918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca17596a22a49089b6f42ab8b96a543'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:37.553689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca17596a22a49089b6f42ab8b96a543'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:37.561538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca17596a22a49089b6f42ab8b96a543'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:37.576462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2449] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:18:37 2022] DELETE /identity/v3/users/cca17596a22a49089b6f42ab8b96a543 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:18:37.581064 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:37.581542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:37.581719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:37.599894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:37.601150 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:37.662851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2450] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:37.672715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:37.674455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:37.731743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3144d-e315-4f55-9073-54bcefdecf93 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:37.735746 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:37.736582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:37.736910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:37.737173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:37.747485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:37.748984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063d375-7a9a-46f8-b0ae-c24de86259b2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:37.759543 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2451] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:38.054811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.056505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:38.070449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0e13-8e78-4e03-868b-c82cd21fb46f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.075879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.076797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.077153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.077345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] PATH_INFO: `/v3/projects/333e2220adfb4ed58ffceb41915e8d90` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.078581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=333e2220adfb4ed58ffceb41915e8d90)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.080997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.132133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333e2220adfb4ed58ffceb41915e8d90', 'request_id': 'req-ab56dac8-293d-4844-8bf9-2b907fea5255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:38.137895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333e2220adfb4ed58ffceb41915e8d90', 'request_id': 'req-ab56dac8-293d-4844-8bf9-2b907fea5255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:38.146288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333e2220adfb4ed58ffceb41915e8d90', 'request_id': 'req-ab56dac8-293d-4844-8bf9-2b907fea5255'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:38.158845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] The token cache is being invalidate because project 333e2220adfb4ed58ffceb41915e8d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:18:38.159351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:38.160141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab56dac8-293d-4844-8bf9-2b907fea5255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:38.162146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2452] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:18:38 2022] DELETE /identity/v3/projects/333e2220adfb4ed58ffceb41915e8d90 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:18:38.166936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.167311 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.167593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.186568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:38.187779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:38.207029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:38.261884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2453] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:18:38.269939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.270234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.270511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.290205 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:38.291431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:38.300760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2454] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:38.305164 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.315529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff770a-dc06-44d1-b393-40d45a89de9b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.320971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.321667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.321958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.322240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.322979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:18:38.323288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.324947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.337100 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2455] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:18:38.341530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.351552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d85205-1af1-4aaa-a7b1-5cbec1cc1536 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.357706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.358503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.358788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.359063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.359763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.361957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.394771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2456] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:18:38 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:18:38.399178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.411920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed0cd7-afd2-4b75-82b7-991035de85a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.467784 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.468493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.468777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.469044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.469737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.471824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.491972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2457] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:18:38 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:18:38.496514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.506958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391bb5-0464-4955-8041-a12cb8a2f541 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.517208 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.517894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.518165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.518441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.519172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.520856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.531589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2458] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:18:38.536616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.553467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e63901-af49-4a66-91bd-4fdc202ac7b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.562610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.563934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.564468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.564770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.565623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.567260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bee63a-3d3e-462c-a26b-a6a2c82c6d3f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.578058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2459] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:18:38.583011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.593577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb42bf4-ad16-4301-aac4-dd5126d70aec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.599834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.600743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.601030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.601285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] PATH_INFO: `/v3/projects/c792871994214bbc86367c22728cbdb1/users/26f1818befba4235913690684e69c6a5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.616879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c792871994214bbc86367c22728cbdb1, user_id=26f1818befba4235913690684e69c6a5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.619561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.635019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2460] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:18:38 2022] PUT /identity/v3/projects/c792871994214bbc86367c22728cbdb1/users/26f1818befba4235913690684e69c6a5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:18:38.647023 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.647351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.647604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.677223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] MFA Rules not processed for user `26f1818befba4235913690684e69c6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:38.678457 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:38.744968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2461] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:38.756636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.758509 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:38.781235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:38.819499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca669a8-8cde-4f14-b958-a0e1f5d8cd34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.823329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.823988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.824279 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.824596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.832991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.834426 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6acc48-6afd-439e-af44-6c1a749bf741 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:38.843157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2462] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:38.870075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:38.896954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:38.941077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c32e28-300f-4733-a6b2-0a770179b58e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:38.946114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:38.946987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:38.947271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:38.947542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] PATH_INFO: `/v3/users/26f1818befba4235913690684e69c6a5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:38.948811 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f1818befba4235913690684e69c6a5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:38.950736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.001388 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f1818befba4235913690684e69c6a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.002048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f1818befba4235913690684e69c6a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.013958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f1818befba4235913690684e69c6a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.014607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f1818befba4235913690684e69c6a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.021442 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f1818befba4235913690684e69c6a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.037351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2463] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:18:38 2022] DELETE /identity/v3/users/26f1818befba4235913690684e69c6a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:18:39.041872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.042188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.042467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.060314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:39.061670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:39.122367 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2464] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:39.132597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.134139 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:39.194466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dab02-c02a-47aa-b3eb-d3547c068b26 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.198557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.199200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.199481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.199753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.209176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.210582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863a588-ceab-4f48-8301-183b2acba1f7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.220040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:39.496935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.498893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:39.515969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1b3fa-0222-4182-b315-f54a1fee2cad None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.521481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.522361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.522650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.522926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] PATH_INFO: `/v3/projects/c792871994214bbc86367c22728cbdb1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.524116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c792871994214bbc86367c22728cbdb1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.526111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.574708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c792871994214bbc86367c22728cbdb1', 'request_id': 'req-c9a597c5-0346-4bc4-8768-5997cadb0d14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.580738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c792871994214bbc86367c22728cbdb1', 'request_id': 'req-c9a597c5-0346-4bc4-8768-5997cadb0d14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.590845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c792871994214bbc86367c22728cbdb1', 'request_id': 'req-c9a597c5-0346-4bc4-8768-5997cadb0d14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.602117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] The token cache is being invalidate because project c792871994214bbc86367c22728cbdb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:18:39.602406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.603065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c9a597c5-0346-4bc4-8768-5997cadb0d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:18:39.604689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2466] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:18:39 2022] DELETE /identity/v3/projects/c792871994214bbc86367c22728cbdb1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:18:39.609720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.610169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.610282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.628451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:39.629663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:39.676729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:39.703853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2467] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:39.711811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.712103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.712431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.731229 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:39.732442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:39.741399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2468] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:39.746586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.762574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad0d7f-5472-4909-b326-f31a5db4603e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.769590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.770241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.770526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.770800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.771838 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:18:39.772350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.774188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.786435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2469] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:18:39.790734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.800789 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3422a9-722d-40b9-aef6-a636555fe6c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.806366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.807012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.807306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.807582 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.808288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.810244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.838623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2470] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:18:39 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:18:39.842648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.853379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f928c7-d9c4-417a-b890-5c82051c72c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.916095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.917387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.917776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.918150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.918976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.921141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.945218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2471] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:18:39 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:18:39.949731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:39.962765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d62df3-1bdb-4185-9398-7c6098a11ab7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:39.968169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:39.968940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:39.969228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:39.969503 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:39.970250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:39.973029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:39.988969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2472] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:18:39.990068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:40.011166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921b085-75df-4a11-8cb5-dd57f4f447e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:40.014884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:40.015572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:40.015861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:40.016143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:40.016919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:40.018582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732fa65-7714-4b56-b69b-7e8cc6db4070 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:40.029862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2473] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:18:40.034443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:40.046232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a707d3-c5a3-431e-a56f-5a6ff4ac99c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:40.051723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:40.052693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:40.052975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:40.053262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] PATH_INFO: `/v3/projects/f3f72e960bee47c48d299b36cc979d65/users/514126f5063b4c6f84013ca75f63f3ff/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:40.055947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:40.072338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f72e960bee47c48d299b36cc979d65, user_id=514126f5063b4c6f84013ca75f63f3ff, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:40.075023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:40.089074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2474] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:18:40 2022] PUT /identity/v3/projects/f3f72e960bee47c48d299b36cc979d65/users/514126f5063b4c6f84013ca75f63f3ff/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:18:40.100656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:40.100946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:40.101220 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:40.134538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] MFA Rules not processed for user `514126f5063b4c6f84013ca75f63f3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:18:40.135973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:40.205806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2475] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:18:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:18:40.216788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:40.218632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:40.279432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856091b0-81c3-464a-b3c1-8dd40e5dda79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:40.285604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:40.286576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:40.286943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:40.287516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:40.300766 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:40.302203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:40.311560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:43.430177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:43.431724 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:43.472202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:43.492033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdff15-005a-4d60-8f8a-14eccb3f8953 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:43.495754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:43.496747 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:43.497031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:43.497304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:43.499266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:43.579203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:43.580850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:43.592125 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2477] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:44.780180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:18:44.781799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:44.794497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035daacf-8fc4-45fc-a47f-2ef9e9f0d0f1 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:18:44.798180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:18:44.798825 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:18:44.799111 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:18:44.799525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:18:44.801340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:18:44.814350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:18:44.815805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:18:44.826618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2478] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:18:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:18:53.587295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=38 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:18:54.822122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:05.209598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:05.222452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c61d-76ff-4531-b7cb-e09818773e97 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:05.227296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:05.227949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:05.228235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:05.228557 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:05.230644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:05.297314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:05.298782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb158f0-787c-4127-841c-e943bf7247cd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:05.308843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2479] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:07.107858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:07.177874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1596-3f45-426b-bf0d-57b322a7654a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:07.182855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:07.183736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:07.184020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:07.184298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] PATH_INFO: `/v3/users/514126f5063b4c6f84013ca75f63f3ff` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:07.185632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514126f5063b4c6f84013ca75f63f3ff)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:07.187560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:07.236339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514126f5063b4c6f84013ca75f63f3ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.237210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514126f5063b4c6f84013ca75f63f3ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.237855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514126f5063b4c6f84013ca75f63f3ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.250195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514126f5063b4c6f84013ca75f63f3ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.260731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514126f5063b4c6f84013ca75f63f3ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.274328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2480] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:07 2022] DELETE /identity/v3/users/514126f5063b4c6f84013ca75f63f3ff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:07.278907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:07.279406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:07.279514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:07.297197 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:07.298472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:07.360075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2481] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:07.370287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:07.372065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:07.432193 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93903d-7574-4e77-88d3-cabb79219840 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:07.436083 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:07.436767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:07.437052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:07.437327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:07.447077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:07.448559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:07.458824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2482] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:19:07.798483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:07.800087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:07.812043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983360-6aa7-46ba-88cc-088ca6be2c77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:07.817321 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:07.818295 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:07.818632 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:07.819020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] PATH_INFO: `/v3/projects/f3f72e960bee47c48d299b36cc979d65` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:07.820289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f72e960bee47c48d299b36cc979d65)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:07.822516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:07.889805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f72e960bee47c48d299b36cc979d65', 'request_id': 'req-c6640bb9-a718-425f-aa0a-6331293e5e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.897675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f72e960bee47c48d299b36cc979d65', 'request_id': 'req-c6640bb9-a718-425f-aa0a-6331293e5e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.904310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f72e960bee47c48d299b36cc979d65', 'request_id': 'req-c6640bb9-a718-425f-aa0a-6331293e5e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.920263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] The token cache is being invalidate because project f3f72e960bee47c48d299b36cc979d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:07.920867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.921847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:07.923843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2483] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:07 2022] DELETE /identity/v3/projects/f3f72e960bee47c48d299b36cc979d65 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:15.561628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:15.571194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee425999-529b-412b-83ad-ceac70e234f9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:15.591437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:15.625500 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:15.626137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:15.626417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:15.626686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:15.628852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:15.693277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:15.694723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:15.705113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:19:17.916834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:19.317585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:19.319266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:19.334099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640bb9-a718-425f-aa0a-6331293e5e49 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:19.338975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:19.339884 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.340177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.340491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] PATH_INFO: `/v3/users/937fff3ec09e4f8a9a9a5fcdf2027a2b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.355815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937fff3ec09e4f8a9a9a5fcdf2027a2b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:19.357906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:19.408958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937fff3ec09e4f8a9a9a5fcdf2027a2b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.409732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937fff3ec09e4f8a9a9a5fcdf2027a2b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.422227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937fff3ec09e4f8a9a9a5fcdf2027a2b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.422887 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937fff3ec09e4f8a9a9a5fcdf2027a2b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.430236 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1771f112-950b-4ebe-9caa-b0b6c45660b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937fff3ec09e4f8a9a9a5fcdf2027a2b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.444153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2485] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:19 2022] DELETE /identity/v3/users/937fff3ec09e4f8a9a9a5fcdf2027a2b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:19.627622 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:19.638795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2c7f-8ef2-42f4-bc56-8a34f3a8f354 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:19.699416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:19.700217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.700530 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.700814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] PATH_INFO: `/v3/projects/8dd05fc495454e9290c277ed4db003d8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.711967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd05fc495454e9290c277ed4db003d8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:19.713926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:19.763169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd05fc495454e9290c277ed4db003d8', 'request_id': 'req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.769021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd05fc495454e9290c277ed4db003d8', 'request_id': 'req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.778343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd05fc495454e9290c277ed4db003d8', 'request_id': 'req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.779487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:19.792941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] The token cache is being invalidate because project 8dd05fc495454e9290c277ed4db003d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:19.793228 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.793961 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0667e6cf-3f1e-4cb4-8c8a-91d760d71133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:19.795542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2486] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:19 2022] DELETE /identity/v3/projects/8dd05fc495454e9290c277ed4db003d8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:19.801058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.801375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.801653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.824495 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:19.826353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:19.896455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2487] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:19.904620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.904946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.905230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.926227 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:19.927455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:19.936573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2488] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:19.940850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:19.952113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc7667-d843-440e-ab48-d0c0e6059126 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:19.958436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:19.959107 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.959375 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.959675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.960424 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:19.960730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:19.962344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:19.978902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2489] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:19:19.980423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:19.989970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be79fa-bd5a-4cb5-a2de-1f72721b9daa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:19.995808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:19.996516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:19.996799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:19.997074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:19.997765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:19.999692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.027375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2490] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:19 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:20.031600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.041787 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e01d28-3d9b-4597-acea-880f5f4dc887 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.094347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:20.108608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.109283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.109564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.109835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.110527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.112507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.136845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2491] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:20 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:20.141298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.153013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c68de-e570-46c6-ab2d-f745bf606229 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.159223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.160344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.160663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.160944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.161691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.163341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.174182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2492] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:20.178658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.189644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e24a93-94eb-4dd7-a55c-49019bdb8502 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.195432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.196098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.196380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.196690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.197421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.199046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256734d-1fc4-4395-8462-563606b8b89f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.209325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2493] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:20.213478 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.223935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab34e2b-b4ac-43ac-ac2b-ac451de8711c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.229109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.229998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.230308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.230588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] PATH_INFO: `/v3/projects/67b6a3283112495ab5c80c3716c8adbb/users/c37acd535a964980865fe2714e9b5eda/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.246978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b6a3283112495ab5c80c3716c8adbb, user_id=c37acd535a964980865fe2714e9b5eda, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.249681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.265317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2494] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:20 2022] PUT /identity/v3/projects/67b6a3283112495ab5c80c3716c8adbb/users/c37acd535a964980865fe2714e9b5eda/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:20.276959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.277253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.277673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.307749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] MFA Rules not processed for user `c37acd535a964980865fe2714e9b5eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:20.308994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:20.381483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2495] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:20.386109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.449428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0374f-86fa-4389-91ef-3504c807ef9e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.454552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.455327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.455703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.455983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.456696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.458592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.485808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2496] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:20 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:20.490004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.501050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf99acd-6f64-4969-8c05-4e76d1e52058 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.561635 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.562637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.563269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.563686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.564773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.567834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.594488 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2497] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:20 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:20.598956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.609529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34589d53-f901-4d96-9df4-2b3bdc69c2a8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.614735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.615393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.615679 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.615956 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.616724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.618324 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.629595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2498] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:20.634205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.644540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8246f00-53ca-4852-be68-4e9d8852baa3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.649565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.650455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.650740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.651122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] PATH_INFO: `/v3/projects/05d1f43edd574b6a918e0c7b459f4af8/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.666326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d1f43edd574b6a918e0c7b459f4af8, user_id=7b01b0f2f79f4cd1b8b1b6673b0b0dbe, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.669037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.686825 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2499] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:20 2022] PUT /identity/v3/projects/05d1f43edd574b6a918e0c7b459f4af8/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:20.692081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.702442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c8e7e-949f-4578-bff6-db85431c441d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.764807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.765405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.765687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.765963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.766692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.768521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4225b-0465-4991-9d89-af63094023c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.773714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:20.780340 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:20.783338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.793930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e7820-8f3c-4794-8c93-edd80d8dc701 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:20.798756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:20.799662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.799945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.800222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] PATH_INFO: `/v3/projects/05d1f43edd574b6a918e0c7b459f4af8/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.802330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d1f43edd574b6a918e0c7b459f4af8, user_id=7b01b0f2f79f4cd1b8b1b6673b0b0dbe, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:20.805051 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:20.815411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2501] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:20 2022] PUT /identity/v3/projects/05d1f43edd574b6a918e0c7b459f4af8/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:20.830037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:20.830369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:20.830767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:20.863519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] MFA Rules not processed for user `7b01b0f2f79f4cd1b8b1b6673b0b0dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:20.864764 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:20.936732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2502] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:20 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:20.947689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:20.949542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:20.960745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:21.007585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0387db-39d2-4f85-8a28-12a62fd2424e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:21.012104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:21.012780 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:21.013059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:21.013332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:21.022968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:21.024349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8af27-d8fd-4e7d-bff1-ce3dddef58e8 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:21.033983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2503] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:20 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:21.184705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:21.254123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc3a1d-7ef4-4880-8cfa-5f4f1e078ec5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:21.259004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:21.259888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:21.260169 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:21.260477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] PATH_INFO: `/v3/users/c37acd535a964980865fe2714e9b5eda` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:21.261690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37acd535a964980865fe2714e9b5eda)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:21.263598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:21.313758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37acd535a964980865fe2714e9b5eda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:21.314476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37acd535a964980865fe2714e9b5eda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:21.315163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37acd535a964980865fe2714e9b5eda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:21.327463 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37acd535a964980865fe2714e9b5eda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:21.337168 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37acd535a964980865fe2714e9b5eda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:21.350311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2504] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:21 2022] DELETE /identity/v3/users/c37acd535a964980865fe2714e9b5eda => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:21.355154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:21.355444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:21.355725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:21.373683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:21.374882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:21.433666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2505] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:21.446599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:21.448433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:21.456432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:21.511481 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf907eb2-51ec-4791-96cd-4c213237b369 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:21.515856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:21.516534 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:21.516816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:21.517097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:21.526613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:21.528038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:21.537946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:22.068170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:22.069728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:22.083985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f94e1d-c8c4-4da0-9bc9-d48ba21b5781 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:22.089793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:22.090918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.091391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.091805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] PATH_INFO: `/v3/projects/67b6a3283112495ab5c80c3716c8adbb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.093511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b6a3283112495ab5c80c3716c8adbb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:22.095998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:22.149195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b6a3283112495ab5c80c3716c8adbb', 'request_id': 'req-4a665473-ce2f-433a-a2eb-f25c603e95ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.154734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b6a3283112495ab5c80c3716c8adbb', 'request_id': 'req-4a665473-ce2f-433a-a2eb-f25c603e95ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.161142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b6a3283112495ab5c80c3716c8adbb', 'request_id': 'req-4a665473-ce2f-433a-a2eb-f25c603e95ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.176504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] The token cache is being invalidate because project 67b6a3283112495ab5c80c3716c8adbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:22.176801 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.177521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.179185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2507] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:22 2022] DELETE /identity/v3/projects/67b6a3283112495ab5c80c3716c8adbb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:22.183466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:22.193329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2ba6-3ab6-444b-a892-711c1780a06c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:22.252005 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:22.253102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.253385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.253658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] PATH_INFO: `/v3/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.254893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b01b0f2f79f4cd1b8b1b6673b0b0dbe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:22.256867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:22.315084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b01b0f2f79f4cd1b8b1b6673b0b0dbe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.315779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b01b0f2f79f4cd1b8b1b6673b0b0dbe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.316440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b01b0f2f79f4cd1b8b1b6673b0b0dbe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.331894 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b01b0f2f79f4cd1b8b1b6673b0b0dbe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.343238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5df1af3e-4718-41ae-a54b-2696f5bb441e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b01b0f2f79f4cd1b8b1b6673b0b0dbe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.357217 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2508] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:22 2022] DELETE /identity/v3/users/7b01b0f2f79f4cd1b8b1b6673b0b0dbe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:22.635451 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:22.637088 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:22.651371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665473-ce2f-433a-a2eb-f25c603e95ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:22.685622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:22.711036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:22.711868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.712150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.712443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] PATH_INFO: `/v3/projects/05d1f43edd574b6a918e0c7b459f4af8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.713689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d1f43edd574b6a918e0c7b459f4af8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:22.715605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:22.771836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d1f43edd574b6a918e0c7b459f4af8', 'request_id': 'req-55758912-7adf-4872-81b7-41c7265c692a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.777716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d1f43edd574b6a918e0c7b459f4af8', 'request_id': 'req-55758912-7adf-4872-81b7-41c7265c692a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.783939 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d1f43edd574b6a918e0c7b459f4af8', 'request_id': 'req-55758912-7adf-4872-81b7-41c7265c692a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.797690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] The token cache is being invalidate because project 05d1f43edd574b6a918e0c7b459f4af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:22.797976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.798711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-55758912-7adf-4872-81b7-41c7265c692a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:22.800314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2509] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:22 2022] DELETE /identity/v3/projects/05d1f43edd574b6a918e0c7b459f4af8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:22.806249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.806576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.806864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.825223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:22.826429 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:22.847840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:22.889608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2510] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:22.897574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.897866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.898140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.917201 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:22.918391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:22.926893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2511] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:22.931189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:22.941550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec4d2-7f41-423f-946a-477b31463715 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:22.947412 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:22.948105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.948389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.948698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.949430 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:22.949737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:22.951390 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:22.963459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2512] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:19:22.968196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:22.978853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d58225-1bf0-4dbd-84c8-bcad3a13f956 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:22.985306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:22.985967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:22.986254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:22.986529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:22.987226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:22.989203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.019572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2513] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:22 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:23.024120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.035619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317f09-d805-4d89-8938-3a933a66e672 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.093562 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.094232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.094515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.094834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.095585 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.097612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.118660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2514] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:23 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:23.123266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.133772 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b5b9f-4dbb-4539-a1da-5a53600e01ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.138908 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.139575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.139863 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.140140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.140911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.142666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.157570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2515] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:23.161756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.172282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0f5f2-cbc7-4d7d-8dfc-36572755c38d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.177350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.178019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.178306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.178589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.179345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.180988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdeb3c-3948-4440-b68b-6a3971a39b0c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.191460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2516] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:23.195706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.206126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141054f7-622a-4d7b-bfc3-2759e060566f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.211031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.211913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.212198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.212484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] PATH_INFO: `/v3/projects/88295ac098e246fa8504058413c41432/users/c16b8f1f016f4e35b912271e9fc8585d/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.227132 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88295ac098e246fa8504058413c41432, user_id=c16b8f1f016f4e35b912271e9fc8585d, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.229861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.244152 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2517] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:23 2022] PUT /identity/v3/projects/88295ac098e246fa8504058413c41432/users/c16b8f1f016f4e35b912271e9fc8585d/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:23.256037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.256335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.256653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.283874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] MFA Rules not processed for user `c16b8f1f016f4e35b912271e9fc8585d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:23.285121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:23.352207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2518] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:23.357562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.423839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f8c3-aca1-4b99-bbda-2b8f53b70a9b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.429563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.430215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.430499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.430768 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.431474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.433526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.437842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:23.461875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2519] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:23 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:23.465960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.477031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94963dd2-a45c-48e5-8323-f2bb3836f507 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.496197 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:23.537774 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.538441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.538720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.538989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.539684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.541624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.572012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2520] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:23 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:23.578269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.590099 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3e4cc-53b7-43a3-a053-7fdce2251230 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.595458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.596134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.596434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.596736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.597481 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.599108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.609890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2521] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:23.614634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.630752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526beb8-33be-44bb-a5be-0862684a25a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.639180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.640584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.640995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.641309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] PATH_INFO: `/v3/projects/cb100302050543ddbb1d0ea2058fd4f1/users/a2f66a65e5eb45fabf6fa0eebebebcc9/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.657211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb100302050543ddbb1d0ea2058fd4f1, user_id=a2f66a65e5eb45fabf6fa0eebebebcc9, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.659852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.674035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2522] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:23 2022] PUT /identity/v3/projects/cb100302050543ddbb1d0ea2058fd4f1/users/a2f66a65e5eb45fabf6fa0eebebebcc9/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:23.677972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.688040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2849-5800-4dab-986e-ea38eff72064 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.746548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.747297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.747587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.747867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.748621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.750224 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452bce7-df55-45a9-bbf4-358e76c96f91 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.760332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2523] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:23.764577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.775157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7c6a1-c7b2-4ae1-bac3-a1c6869c4290 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.780144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.781450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.781768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.782086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] PATH_INFO: `/v3/projects/cb100302050543ddbb1d0ea2058fd4f1/users/a2f66a65e5eb45fabf6fa0eebebebcc9/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.784148 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb100302050543ddbb1d0ea2058fd4f1, user_id=a2f66a65e5eb45fabf6fa0eebebebcc9, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:23.786827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:23.797184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2524] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:23 2022] PUT /identity/v3/projects/cb100302050543ddbb1d0ea2058fd4f1/users/a2f66a65e5eb45fabf6fa0eebebebcc9/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:23.807213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.807511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.807789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:23.836329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] MFA Rules not processed for user `a2f66a65e5eb45fabf6fa0eebebebcc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:23.837682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:23.917813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2525] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:23.929532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:23.932724 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:23.992464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7b9f0-38db-41fd-b27c-0c496c8a177b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:23.996476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:23.997181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:23.997411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:23.997690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:24.007654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:24.009125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d106bc-1dfc-4f07-b682-215aacb09a82 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:24.019131 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2526] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:24.428885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:24.489492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:24.496400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b30472-5363-4626-854f-08dedeaa9ada None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:24.503729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:24.504616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:24.504901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:24.505176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] PATH_INFO: `/v3/users/c16b8f1f016f4e35b912271e9fc8585d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:24.506522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16b8f1f016f4e35b912271e9fc8585d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:24.508510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:24.559483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16b8f1f016f4e35b912271e9fc8585d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:24.560151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16b8f1f016f4e35b912271e9fc8585d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:24.571860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16b8f1f016f4e35b912271e9fc8585d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:24.572606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16b8f1f016f4e35b912271e9fc8585d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:24.583483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16b8f1f016f4e35b912271e9fc8585d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:24.601699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2527] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:24 2022] DELETE /identity/v3/users/c16b8f1f016f4e35b912271e9fc8585d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:24.606355 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:24.606849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:24.606991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:24.625560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:24.627133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:24.687916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2528] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:24.697834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:24.699330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:24.763600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843080df-056e-4959-825c-a087aab183e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:24.768348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:24.769046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:24.769331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:24.769608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:24.780263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:24.781752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:24.792343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2529] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:25.084280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:25.085967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:25.104119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991b998-fc7d-4303-bdc1-5d8005993568 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:25.110498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:25.111482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.111775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.112149 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] PATH_INFO: `/v3/projects/88295ac098e246fa8504058413c41432` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.113486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88295ac098e246fa8504058413c41432)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:25.115417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:25.128226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:25.165827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88295ac098e246fa8504058413c41432', 'request_id': 'req-916783d3-d529-48f8-a451-54a89a504c86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.171520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88295ac098e246fa8504058413c41432', 'request_id': 'req-916783d3-d529-48f8-a451-54a89a504c86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.180935 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88295ac098e246fa8504058413c41432', 'request_id': 'req-916783d3-d529-48f8-a451-54a89a504c86'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.192020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] The token cache is being invalidate because project 88295ac098e246fa8504058413c41432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:25.192305 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.192954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.194502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2530] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:25 2022] DELETE /identity/v3/projects/88295ac098e246fa8504058413c41432 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:25.198681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:25.208843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0ede-7077-4eaa-be96-489822264a05 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:25.280504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:25.281310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.281597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.281874 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] PATH_INFO: `/v3/users/a2f66a65e5eb45fabf6fa0eebebebcc9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.283640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f66a65e5eb45fabf6fa0eebebebcc9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:25.285701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:25.307868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:25.342576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f66a65e5eb45fabf6fa0eebebebcc9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.343592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f66a65e5eb45fabf6fa0eebebebcc9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.357448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f66a65e5eb45fabf6fa0eebebebcc9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.358090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f66a65e5eb45fabf6fa0eebebebcc9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.365070 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c7d7a9a8-8486-4b70-a710-3a2003a842c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f66a65e5eb45fabf6fa0eebebebcc9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.382184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2531] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:25 2022] DELETE /identity/v3/users/a2f66a65e5eb45fabf6fa0eebebebcc9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:25.661939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:25.663400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:25.676938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916783d3-d529-48f8-a451-54a89a504c86 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:25.736743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:25.737639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.737920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.738199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] PATH_INFO: `/v3/projects/cb100302050543ddbb1d0ea2058fd4f1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.739393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb100302050543ddbb1d0ea2058fd4f1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:25.741360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:25.797953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb100302050543ddbb1d0ea2058fd4f1', 'request_id': 'req-5dc4ed15-a601-46a5-8243-4057fe5c6619'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.803602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb100302050543ddbb1d0ea2058fd4f1', 'request_id': 'req-5dc4ed15-a601-46a5-8243-4057fe5c6619'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.813224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb100302050543ddbb1d0ea2058fd4f1', 'request_id': 'req-5dc4ed15-a601-46a5-8243-4057fe5c6619'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.824484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] The token cache is being invalidate because project cb100302050543ddbb1d0ea2058fd4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:25.824771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.825394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5dc4ed15-a601-46a5-8243-4057fe5c6619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:25.826958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2532] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:25 2022] DELETE /identity/v3/projects/cb100302050543ddbb1d0ea2058fd4f1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:25.831930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.832441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.832542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.865734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:25.866958 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:25.926676 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2533] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:25.935682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502257-8031-4499-b742-ed96df3f49ac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.935972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502257-8031-4499-b742-ed96df3f49ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.936249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03502257-8031-4499-b742-ed96df3f49ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.956805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-03502257-8031-4499-b742-ed96df3f49ac None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:25.958017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03502257-8031-4499-b742-ed96df3f49ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:25.970054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2534] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:25.976473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:25.989385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b315a5-1fac-4d88-826c-796c36662807 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:25.996074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:25.996779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:25.997048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:25.997320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:25.998056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:25.998367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:25.999993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.012930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2535] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:19:26.017508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502257-8031-4499-b742-ed96df3f49ac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.027498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03502257-8031-4499-b742-ed96df3f49ac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.033246 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.033902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.034186 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.034456 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.035156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.037121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.065948 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2536] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:26 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:26.070108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.080174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05684b4-82c9-4a02-b375-72229fa3554f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.144061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.144766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.145061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.145348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.146048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.148009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.172556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2537] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:26 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:26.176788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.182577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:26.188521 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aeb0dc-e039-4d02-a280-8e7e7c26dfd1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.194831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.195741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.196038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.196451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.197235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.199019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.209954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2538] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:19:26.214076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.225177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1cc5e1-c3eb-4bc7-997f-96689cb67767 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.230656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.231332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.231621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.231906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.232684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.234303 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2072d1-748f-47f5-8a23-809c1aefa1b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.244893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2539] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:26.250628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.261192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb6b9c-b796-4fe7-8933-a89c10fc50ab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.266000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.266896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.267194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.267647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] PATH_INFO: `/v3/projects/9cabbd5f55824648a904006c9fb98b23/users/02eb1c32de2044dc918b6d92dc4cea05/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.282958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cabbd5f55824648a904006c9fb98b23, user_id=02eb1c32de2044dc918b6d92dc4cea05, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.285682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.300539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2540] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:26 2022] PUT /identity/v3/projects/9cabbd5f55824648a904006c9fb98b23/users/02eb1c32de2044dc918b6d92dc4cea05/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:19:26.310840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.311134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.311407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.319112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:26.340913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] MFA Rules not processed for user `02eb1c32de2044dc918b6d92dc4cea05`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:26.342133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:26.405945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2541] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:26.427010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.429094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:26.494037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4f2b0-be32-4ab2-b400-c3772a119a4b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.498784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.499428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.499707 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.499981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.509742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.511199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8fe67-d54b-487f-8905-343550e29328 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.520558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2542] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:26.539636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.605711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d7593-68b2-4155-989d-faf077468669 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.611552 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.612469 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.612779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.613049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] PATH_INFO: `/v3/users/02eb1c32de2044dc918b6d92dc4cea05` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.614346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02eb1c32de2044dc918b6d92dc4cea05)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.616267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.665287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02eb1c32de2044dc918b6d92dc4cea05'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:26.665959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02eb1c32de2044dc918b6d92dc4cea05'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:26.678756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02eb1c32de2044dc918b6d92dc4cea05'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:26.679425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02eb1c32de2044dc918b6d92dc4cea05'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:26.687163 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02eb1c32de2044dc918b6d92dc4cea05'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:26.702061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2543] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:26 2022] DELETE /identity/v3/users/02eb1c32de2044dc918b6d92dc4cea05 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:26.706900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.707383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.707482 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.725268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:26.726464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:26.794253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2544] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:26.806301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:26.807972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:26.823718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:26.868083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f4c2-2ad0-4b4d-9fab-599e5c3fedc2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:26.873068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:26.873722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:26.874004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:26.874278 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:26.884812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:26.886246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d9920-2048-4a33-87d8-4a9ec1cf7301 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:26.896315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2545] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:19:27.192467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.193958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:27.207033 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4126d-3052-4476-a193-71ca80b58590 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.212741 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.214345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.214639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.214920 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] PATH_INFO: `/v3/projects/9cabbd5f55824648a904006c9fb98b23` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.216321 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cabbd5f55824648a904006c9fb98b23)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.218268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.268341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cabbd5f55824648a904006c9fb98b23', 'request_id': 'req-76caa636-27aa-4181-9773-98e2be03ff0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:27.274045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cabbd5f55824648a904006c9fb98b23', 'request_id': 'req-76caa636-27aa-4181-9773-98e2be03ff0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:27.284119 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cabbd5f55824648a904006c9fb98b23', 'request_id': 'req-76caa636-27aa-4181-9773-98e2be03ff0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:27.285411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:27.298139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] The token cache is being invalidate because project 9cabbd5f55824648a904006c9fb98b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:27.298460 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:27.299097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76caa636-27aa-4181-9773-98e2be03ff0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:27.300689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2546] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:27 2022] DELETE /identity/v3/projects/9cabbd5f55824648a904006c9fb98b23 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:27.305996 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.306283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.306561 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.324851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:27.326052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:27.390395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2547] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:27.398999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.399294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.399574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.419841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:27.421122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:27.431125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2548] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:27.435907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.446712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad00da3-234f-4053-8f01-e05cf502f274 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.452934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.453610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.453906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.454181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.455032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:27.455353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.457035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.469687 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2549] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:19:27.474298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.484113 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6b9b5-4961-4899-b5a9-a4fd5720d3c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.489626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.490292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.490578 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.490851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.491558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.493532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.526446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2550] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:27 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:27.530633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.541173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082efa5e-3d94-4c42-97c8-cb3fd2e1624b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.597115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.597773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.598054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.598324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.599010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.601003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.627151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2551] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:27 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:27.631437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.642115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeaf30-94c2-4d54-8691-b91622bd6c70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.647094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.647754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.648040 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.648339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.649104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.650714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.661295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2552] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:19:27.665094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.675539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d1d77-bdc9-480c-9d64-3089542acbe2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.681519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.682211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.682499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.682805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.683576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.685309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d33a1c-9922-43f3-a150-a0142784b11c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.696937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2553] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:27.701174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.712090 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f221a80-a155-47b8-8c2a-000bbb00de95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.717857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.718829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.719117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.719399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] PATH_INFO: `/v3/projects/3465e0c74eb546b6aa0318cb1fa6685a/users/0d9ddc7a5c35446c9d1ef4f8e3f9b31a/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.734603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3465e0c74eb546b6aa0318cb1fa6685a, user_id=0d9ddc7a5c35446c9d1ef4f8e3f9b31a, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.737350 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.755238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2554] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:27 2022] PUT /identity/v3/projects/3465e0c74eb546b6aa0318cb1fa6685a/users/0d9ddc7a5c35446c9d1ef4f8e3f9b31a/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:27.766553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.766841 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.767102 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.797928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] MFA Rules not processed for user `0d9ddc7a5c35446c9d1ef4f8e3f9b31a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:27.799156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:27.813169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:27.870352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2555] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:27.882194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:27.888368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:27.959301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aae48-cdbf-4ec9-b5f5-c3d212b0af72 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:27.963691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:27.964335 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:27.964653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:27.964926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:27.974102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:27.975520 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f0f2ac-ba44-4ce9-88bc-93c0c56ce02c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:27.984461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2556] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:31.068267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:31.134038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4397ecf-808a-4ff2-8f10-b7144dccdf4e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:31.139190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:31.140067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.140348 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.140648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] PATH_INFO: `/v3/users/0d9ddc7a5c35446c9d1ef4f8e3f9b31a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.141890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9ddc7a5c35446c9d1ef4f8e3f9b31a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:31.143974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:31.198745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9ddc7a5c35446c9d1ef4f8e3f9b31a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.199395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9ddc7a5c35446c9d1ef4f8e3f9b31a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.212596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9ddc7a5c35446c9d1ef4f8e3f9b31a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.213274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9ddc7a5c35446c9d1ef4f8e3f9b31a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.220251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9ddc7a5c35446c9d1ef4f8e3f9b31a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.235454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2557] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:31 2022] DELETE /identity/v3/users/0d9ddc7a5c35446c9d1ef4f8e3f9b31a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:31.240657 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.241181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.241712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.260439 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:31.261689 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:31.322270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2558] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:31.332363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:31.334095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:31.370785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:31.395269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c618a0-9655-4d16-a7d7-04a14f49340a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:31.399116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:31.399761 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.400034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.400306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.409806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:31.411284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271ce3c-ded9-4d49-acab-7417f5cc2e1c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:31.420741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2559] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:31.710873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:31.712497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:31.726528 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d25cc7-cd12-4f50-8bfd-6ad6d786bcac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:31.732006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:31.732906 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.733189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.733466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] PATH_INFO: `/v3/projects/3465e0c74eb546b6aa0318cb1fa6685a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.734850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3465e0c74eb546b6aa0318cb1fa6685a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:31.736950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:31.770580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:31.786199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3465e0c74eb546b6aa0318cb1fa6685a', 'request_id': 'req-e74208b7-406b-4499-912c-035fc4c17212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.791589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3465e0c74eb546b6aa0318cb1fa6685a', 'request_id': 'req-e74208b7-406b-4499-912c-035fc4c17212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.800756 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3465e0c74eb546b6aa0318cb1fa6685a', 'request_id': 'req-e74208b7-406b-4499-912c-035fc4c17212'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.811967 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] The token cache is being invalidate because project 3465e0c74eb546b6aa0318cb1fa6685a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:31.812259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.813000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e74208b7-406b-4499-912c-035fc4c17212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:31.814799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2560] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:31 2022] DELETE /identity/v3/projects/3465e0c74eb546b6aa0318cb1fa6685a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:31.820261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.820605 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.820886 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.838614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:31.839732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:31.899472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2561] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:31.907410 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.907702 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.908191 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.928580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:31.929798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:31.938869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2562] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:31.943413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:31.953361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe073f-0bf8-4d73-b4cb-8c600f396a45 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:31.958433 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:31.959091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.959376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.959637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:31.960381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:31.960727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:31.962349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:31.974893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2563] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:19:31.979896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:31.992993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887f02-db47-4251-a240-ec5a98c6ac06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:31.998783 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:31.999459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:31.999736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:31.999994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.000728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.002896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.030813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2564] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:31 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:32.034853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:32.044933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2e79-1581-48cf-8160-29f021fd223d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:32.104931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:32.105633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.105753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.106020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.106723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.108713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.133533 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2565] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:32 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:32.137718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:32.148379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b5cc-9712-4581-80d4-256af1b15589 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:32.153285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:32.153957 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.154243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.154518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.155257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.156886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.167806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2566] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:19:32.171937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:32.182744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035ee11-377d-4f78-b38f-9fa6f8978568 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:32.188147 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:32.188845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.189364 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.189836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.190570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.192182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42610508-0200-4c9b-807e-b34a3794ddbd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.202612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2567] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:32.206876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:32.217109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8cd2c-3704-4e11-ace2-feee9b07fc5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:32.222124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:32.223000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.223286 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.223557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] PATH_INFO: `/v3/projects/19f2e5a7cdbe418189c25c72448252ab/users/4a563644bdf946038482e54b61c6708b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.238437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f2e5a7cdbe418189c25c72448252ab, user_id=4a563644bdf946038482e54b61c6708b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.241219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.257294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2568] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:32 2022] PUT /identity/v3/projects/19f2e5a7cdbe418189c25c72448252ab/users/4a563644bdf946038482e54b61c6708b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:32.267767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.268057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.268503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.300867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] MFA Rules not processed for user `4a563644bdf946038482e54b61c6708b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:32.302123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:32.333257 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:32.379056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2569] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:32.391467 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:32.394008 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:32.462612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4c0d2-f34e-4b68-88c4-a4514f531deb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:32.467056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:32.467729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:32.468015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:32.468289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:32.477255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:32.478721 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:32.488100 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2570] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:42.375310 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=14 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:42.475274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:42.477407 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:42.483902 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:42.545654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbc0d9-2bb9-4f28-864a-361bc7633260 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:42.549696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:42.550413 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:42.550696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:42.550974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:42.552926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:42.645322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:42.646844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:42.666706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2571] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:46.188378 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:46.189892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:46.290067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e3d20-db70-4bdb-ba9a-f1b9bdccf3ec service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:46.296491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:46.297296 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:46.297699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:46.298072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:46.300162 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:46.316166 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:46.318306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:46.366155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:47.931525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:47.941639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70afe-4fa3-4db9-9c4c-5a0133648f5e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:47.946356 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:47.947019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:47.947299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:47.947567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:47.949493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:48.040600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:48.044718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:48.056708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:48.066496 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2573] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:51.879833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:51.881428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:51.942871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8eca8-3780-4740-85ba-41ba4ae9c361 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:51.946855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:51.947516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:51.947805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:51.948086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:51.950104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:52.018056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:52.019512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:52.030418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2574] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:55.945889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:55.947979 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:56.009771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:56.012210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:56.028243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089d774-b4e4-420b-a2b8-dc9bc4721c2f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.036157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.037328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.037807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.038139 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] PATH_INFO: `/v3/users/b778c9e1d53c44edb54b0c94c458e20c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.039465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b778c9e1d53c44edb54b0c94c458e20c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.041862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.060706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41549-10a7-4e36-a322-c78f8dad0c3f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.068151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.068889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.069110 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.069334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.073937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:56.093298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.095749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.108328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2575] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:19:56.108845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:56.136593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b778c9e1d53c44edb54b0c94c458e20c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.137458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b778c9e1d53c44edb54b0c94c458e20c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.138280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b778c9e1d53c44edb54b0c94c458e20c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.169772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b778c9e1d53c44edb54b0c94c458e20c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.186009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-513d36bd-203a-4590-afdb-269bb84144ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b778c9e1d53c44edb54b0c94c458e20c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.208459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2576] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:19:56 2022] DELETE /identity/v3/users/b778c9e1d53c44edb54b0c94c458e20c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:56.466419 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:56.482169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd1bdb-6084-4720-adac-78879547d95d service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.566963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.567866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.568210 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.568453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] PATH_INFO: `/v3/projects/5badbc0000cf41beb6fd192462d4eda8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.569821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5badbc0000cf41beb6fd192462d4eda8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.572012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.622949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5badbc0000cf41beb6fd192462d4eda8', 'request_id': 'req-ec249f4e-e58a-4a03-9a47-e445b7ca7108'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.628639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5badbc0000cf41beb6fd192462d4eda8', 'request_id': 'req-ec249f4e-e58a-4a03-9a47-e445b7ca7108'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.637912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5badbc0000cf41beb6fd192462d4eda8', 'request_id': 'req-ec249f4e-e58a-4a03-9a47-e445b7ca7108'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.652648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] The token cache is being invalidate because project 5badbc0000cf41beb6fd192462d4eda8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:19:56.653029 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.653722 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec249f4e-e58a-4a03-9a47-e445b7ca7108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:19:56.655380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2577] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:19:56 2022] DELETE /identity/v3/projects/5badbc0000cf41beb6fd192462d4eda8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:56.660918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.661231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.661512 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.679417 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:56.680682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:56.745532 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2578] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:19:56.754189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.754712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.754995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.776563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:56.777875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:56.787021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2579] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:56.790980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:56.802135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f93b6d-b36f-47e7-b158-08fa027a21f3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.807868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.808631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.808917 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.809220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.810069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:19:56.810298 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.811984 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.824445 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2580] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:19:56.829115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:56.839695 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ac173-35e5-41fe-b61c-1c7ec5bd762d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.845901 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.846623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.846910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.847191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.847930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.849989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.854359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:56.881750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2581] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:19:56 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:19:56.885725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:56.896755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1133-e30e-4d18-850d-4b3e23b12381 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:56.956188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:56.956900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:56.957181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:56.957458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:56.958160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:56.960084 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:56.985198 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2582] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:19:56 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:19:56.990839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:57.005900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183647f-0b6a-4fe0-8380-6a732dd8aaab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:57.011237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:57.011911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:57.012194 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:57.012501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:57.013234 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:57.014848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:57.025617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2583] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:57.029737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:57.041031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f032a-6fe1-4c45-9deb-871f857db216 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:57.047270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:57.048109 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:57.048552 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:57.048840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:57.049639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:57.051147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df135a-820d-4b16-be89-3914dedea709 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:57.061608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2584] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:19:57.066034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:57.077376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b17ce-e2b8-41f6-a0ae-6a20df5b7cc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:57.082499 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:57.083391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:57.083669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:57.083946 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] PATH_INFO: `/v3/projects/e916a2b4aaec4fb0b406e9e0a2ec1ff5/users/0f35b7ee19f64c34817a8aabce53283f/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:57.102555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e916a2b4aaec4fb0b406e9e0a2ec1ff5, user_id=0f35b7ee19f64c34817a8aabce53283f, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:57.106907 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:57.127253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2585] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:19:57 2022] PUT /identity/v3/projects/e916a2b4aaec4fb0b406e9e0a2ec1ff5/users/0f35b7ee19f64c34817a8aabce53283f/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:19:57.138123 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:57.138533 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:57.138801 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:57.165023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:19:57.169085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] MFA Rules not processed for user `0f35b7ee19f64c34817a8aabce53283f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:19:57.170816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:57.238295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2586] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:19:57.250559 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:19:57.253608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:19:57.334810 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84337-ed98-4a0e-8ae5-86d55b702d6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:19:57.341021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:19:57.341675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:19:57.341964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:19:57.342243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:19:57.350528 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:19:57.352191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:19:57.361549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:19:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:07.235182 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=26 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:07.357370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:12.059015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:12.060523 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:12.125144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144cb53-5541-448f-9e87-e18f830d2db4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:12.129021 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:12.129708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:12.129997 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:12.130280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:12.132092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:12.198449 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:12.199940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbc948-c5cc-4618-b2d9-69d71a6019fd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:12.210738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2588] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:13.217485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:13.230453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62df57-e415-4e7e-9963-0d6113299765 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:13.235899 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:13.237148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:13.237707 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:13.238181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] PATH_INFO: `/v3/users/0f35b7ee19f64c34817a8aabce53283f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:13.239751 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f35b7ee19f64c34817a8aabce53283f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:13.241961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:13.310326 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f35b7ee19f64c34817a8aabce53283f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.311057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f35b7ee19f64c34817a8aabce53283f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.323460 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f35b7ee19f64c34817a8aabce53283f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.324158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f35b7ee19f64c34817a8aabce53283f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.331617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f35b7ee19f64c34817a8aabce53283f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.345697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2589] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:13 2022] DELETE /identity/v3/users/0f35b7ee19f64c34817a8aabce53283f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:13.350243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:13.350760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:13.350872 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:13.368078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:13.369709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:13.435519 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2590] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:13.446266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:13.447885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:13.512928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a52eb-937e-4416-b666-157600f89073 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:13.516971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:13.517624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:13.517933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:13.518211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:13.528644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:13.530156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e221189-713b-4711-8a3f-62faa2b43df7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:13.540382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2591] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:13.853974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:13.855621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:13.872160 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123bd01-c89a-4335-acf2-fa0e952ae4db None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:13.878853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:13.880288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:13.880826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:13.881499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] PATH_INFO: `/v3/projects/e916a2b4aaec4fb0b406e9e0a2ec1ff5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:13.883264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e916a2b4aaec4fb0b406e9e0a2ec1ff5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:13.885919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:13.951953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:13.955295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e916a2b4aaec4fb0b406e9e0a2ec1ff5', 'request_id': 'req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.962943 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e916a2b4aaec4fb0b406e9e0a2ec1ff5', 'request_id': 'req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.973167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e916a2b4aaec4fb0b406e9e0a2ec1ff5', 'request_id': 'req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.987245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] The token cache is being invalidate because project e916a2b4aaec4fb0b406e9e0a2ec1ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:13.987626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.988302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b46d7e6-91a1-4b46-9b56-100ee242a9b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:13.992525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2592] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:13 2022] DELETE /identity/v3/projects/e916a2b4aaec4fb0b406e9e0a2ec1ff5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:13.995454 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:13.995752 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:13.996036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.015045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:14.016282 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:14.099466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2593] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:14.107337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.107630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.107989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.129996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:14.131266 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:14.151183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2594] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:14.153097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.167157 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8528346-f3a7-4fc7-b5c7-e1b091d69d13 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.172095 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.172799 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.173083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.174182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.175036 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:14.175344 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.177179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.189822 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2595] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:20:14.194536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.205173 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291cd3c-bf34-4fbc-922d-801c16ec2cbd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.210434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.211201 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.211603 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.212044 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.212604 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.215549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.257981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2596] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:14 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:14.262539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.268109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:14.274666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce62b6-4c63-4362-bae2-529ba1627ba2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.353490 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.354182 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.354462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.354812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.355432 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.357544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.378355 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2597] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:14 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:14.382559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.393327 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc846a56-1f85-4218-840b-48b170307c22 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.399232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.400048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.400357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.400675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.401408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.403210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.414684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2598] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:14.418880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.429066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e60332-2b8e-452a-af81-7c501a34609a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.434518 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.435199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.435497 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.435756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.436525 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.438153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a77a-52b3-4b1d-acb9-729104b78644 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.448592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2599] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:14.453151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.465631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3f8f-a661-4d8c-9f7e-0edeb3c0e518 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.470708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.471612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.472001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.472431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] PATH_INFO: `/v3/projects/041f95c615a142acad774b807fc2fff8/users/6cac4e7f0e1047cf9fa7edc10fa83480/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.490440 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=041f95c615a142acad774b807fc2fff8, user_id=6cac4e7f0e1047cf9fa7edc10fa83480, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.495730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.512098 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2600] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:14 2022] PUT /identity/v3/projects/041f95c615a142acad774b807fc2fff8/users/6cac4e7f0e1047cf9fa7edc10fa83480/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:14.523453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.523826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.524172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.553237 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] MFA Rules not processed for user `6cac4e7f0e1047cf9fa7edc10fa83480`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:14.554515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:14.621748 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2601] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:14.634695 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:14.636848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:14.770287 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7c802-86b2-4500-9294-f543cab5c9ed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:14.777649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:14.778709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:14.779030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:14.779311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:14.788209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:14.789850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11411ac-054b-497e-92e6-626593bcaefe service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:14.799178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:15.226297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:15.304093 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b40451-8218-4b32-bac9-9f91dfc13165 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:15.308774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:15.309788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:15.310158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:15.310504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] PATH_INFO: `/v3/users/6cac4e7f0e1047cf9fa7edc10fa83480` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:15.311770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cac4e7f0e1047cf9fa7edc10fa83480)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:15.313815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:15.329091 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:15.381587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cac4e7f0e1047cf9fa7edc10fa83480'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.382320 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cac4e7f0e1047cf9fa7edc10fa83480'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.402985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cac4e7f0e1047cf9fa7edc10fa83480'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.403691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cac4e7f0e1047cf9fa7edc10fa83480'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.411963 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cac4e7f0e1047cf9fa7edc10fa83480'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.436604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2603] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:15 2022] DELETE /identity/v3/users/6cac4e7f0e1047cf9fa7edc10fa83480 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:15.441839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:15.442904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:15.443376 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:15.464719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:15.466295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:15.544595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:15.576649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2604] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:15.585034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:15.587293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:15.650078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a865-18e1-4b4f-a27a-832445de6e28 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:15.653931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:15.654159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:15.654592 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:15.654885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:15.655165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:15.655598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:15.665366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:15.666861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:15.668069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffae8a-ef2c-491c-88ef-1e43bac72a70 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:15.673771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:15.674681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:15.675016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:15.675348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] PATH_INFO: `/v3/users/5bb9100dcf2a4522840d726d800fe013` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:15.676485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2605] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:15.676638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb9100dcf2a4522840d726d800fe013)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:15.678550 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:15.729555 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb9100dcf2a4522840d726d800fe013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.730267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb9100dcf2a4522840d726d800fe013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.730975 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb9100dcf2a4522840d726d800fe013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.745042 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb9100dcf2a4522840d726d800fe013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.753953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb9100dcf2a4522840d726d800fe013'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:15.766844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2606] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:15 2022] DELETE /identity/v3/users/5bb9100dcf2a4522840d726d800fe013 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:15.987045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:15.989105 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:15.996232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:15.997726 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.007062 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e01176-09e6-486c-80c0-07086bce2efc service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.073153 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.074337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.074599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.074912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] PATH_INFO: `/v3/projects/041f95c615a142acad774b807fc2fff8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.076266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=041f95c615a142acad774b807fc2fff8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.078346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.090757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94894-9f34-455c-9b57-f4f4b40bb417 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.095454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.096348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.096851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.097051 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] PATH_INFO: `/v3/projects/febf273149c744e08e0f7ea65e44e0d9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.098451 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=febf273149c744e08e0f7ea65e44e0d9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.100366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.117515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.129593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041f95c615a142acad774b807fc2fff8', 'request_id': 'req-d0bdab16-e469-4619-92ac-6eb6eb69f74c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.135487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041f95c615a142acad774b807fc2fff8', 'request_id': 'req-d0bdab16-e469-4619-92ac-6eb6eb69f74c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.141651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041f95c615a142acad774b807fc2fff8', 'request_id': 'req-d0bdab16-e469-4619-92ac-6eb6eb69f74c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.154289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febf273149c744e08e0f7ea65e44e0d9', 'request_id': 'req-13baed68-74e3-4931-9d22-c0f154fbae7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.158789 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] The token cache is being invalidate because project 041f95c615a142acad774b807fc2fff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:16.159171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.159957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d0bdab16-e469-4619-92ac-6eb6eb69f74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.160356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febf273149c744e08e0f7ea65e44e0d9', 'request_id': 'req-13baed68-74e3-4931-9d22-c0f154fbae7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.161947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2607] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:15 2022] DELETE /identity/v3/projects/041f95c615a142acad774b807fc2fff8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:16.166714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea40b-b9a1-46c0-9e56-1d4fb8b14c31 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.167038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea40b-b9a1-46c0-9e56-1d4fb8b14c31 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.167269 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea40b-b9a1-46c0-9e56-1d4fb8b14c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.172861 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febf273149c744e08e0f7ea65e44e0d9', 'request_id': 'req-13baed68-74e3-4931-9d22-c0f154fbae7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.181769 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.186866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fa9ea40b-b9a1-46c0-9e56-1d4fb8b14c31 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.187012 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] The token cache is being invalidate because project febf273149c744e08e0f7ea65e44e0d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:16.187567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.188557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13baed68-74e3-4931-9d22-c0f154fbae7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:16.188717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fa9ea40b-b9a1-46c0-9e56-1d4fb8b14c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.190540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2608] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:15 2022] DELETE /identity/v3/projects/febf273149c744e08e0f7ea65e44e0d9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:16.195590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c9e7-1ac0-4a88-814e-e4bb4b69bf01 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.195875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c9e7-1ac0-4a88-814e-e4bb4b69bf01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.196151 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c9e7-1ac0-4a88-814e-e4bb4b69bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.218502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca74c9e7-1ac0-4a88-814e-e4bb4b69bf01 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.219725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca74c9e7-1ac0-4a88-814e-e4bb4b69bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.277193 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2609] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:16.281646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2610] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:16.287144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.287503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.287971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.289004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.289303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.289587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.307714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.308495 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.309042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.309723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.317604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2611] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:16.318960 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2612] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:16.323050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.324687 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.332875 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160a7d1-85bd-43af-a1f4-31b19c87e760 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.333850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f89a8-e199-4dbe-8e32-a03dcd3ef911 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.337541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.338199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.338374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.338501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.338744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.339019 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.339302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.339486 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:16.339647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.339795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.340291 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:16.340628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.341466 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.342200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.353405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2613] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:20:16.354742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2614] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:20:16.357769 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.360785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.374583 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b60e1-a345-4ece-8055-6a4202a530f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.374786 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfc2b7-01f1-43fd-91e0-ac393e226970 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.380081 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.380868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.381148 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.381263 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.381472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.381827 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.382147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.382259 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.382424 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.383122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.384087 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.385072 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.414893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2615] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:16.418719 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2616] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:16.421842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.422898 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.487815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb655ba-61b7-40a5-b6b0-8ffcfd6c88a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.492172 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f208f7-7300-427a-9808-d1d3df1638a7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.492677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.493392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.493673 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.493950 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.495016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.496964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.497225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.497903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.498186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.498494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.499293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.501250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.518628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2617] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:16.521271 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2618] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:16.522816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.525155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.532682 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfb73c-c182-4264-b671-d3c19678b9da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.535075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab663a28-5649-472b-9215-b344acb81367 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.537855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.538518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.538808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.539115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.539881 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.539999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.540694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.541043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.541383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.541670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.542274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.543947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.550187 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.552611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2619] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:16.556566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.558409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2620] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:16.559222 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.567185 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e26f8c-6f21-4ccd-b3e7-b3164124c2bb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.570246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cf7b5-287f-4286-8693-03b466b1247e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.572010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.572720 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.573009 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.573295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.574022 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.575669 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.576405 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.577090 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.577371 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.577644 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.578370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.579986 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.587115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2621] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:16.590749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2622] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:16.591600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.594652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.601870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9762f0f-10d1-463e-8401-ae0907d5b302 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.604384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14737d6-ad5c-4d78-8b9a-32a63fb14f58 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.606570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.607476 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.607758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.608055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] PATH_INFO: `/v3/projects/d2c9067b02ee4ad988b7a85806a7fdd2/users/16929edc079c45bb90de1c6b576abb24/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.609001 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.609883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.610261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.610443 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] PATH_INFO: `/v3/projects/5d56e45a575b4af58dd61c673aeaaf45/users/f35848201c8c4c3b8f0d71d5e1ade7c7/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.622740 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c9067b02ee4ad988b7a85806a7fdd2, user_id=16929edc079c45bb90de1c6b576abb24, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.625406 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d276fcc-5837-4041-829b-62426675bc21 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.625554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d56e45a575b4af58dd61c673aeaaf45, user_id=f35848201c8c4c3b8f0d71d5e1ade7c7, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.628127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e061f62-0777-4c8b-99af-ed9c58159517 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.636684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.641389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2623] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:16 2022] PUT /identity/v3/projects/5d56e45a575b4af58dd61c673aeaaf45/users/f35848201c8c4c3b8f0d71d5e1ade7c7/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:16.649365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2624] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:16 2022] PUT /identity/v3/projects/d2c9067b02ee4ad988b7a85806a7fdd2/users/16929edc079c45bb90de1c6b576abb24/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:16.651835 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.652166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.652479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.659877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.660202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.660522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.681661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] MFA Rules not processed for user `f35848201c8c4c3b8f0d71d5e1ade7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.684030 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.692954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] MFA Rules not processed for user `16929edc079c45bb90de1c6b576abb24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:16.694485 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:16.755274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2625] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:16.760158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.761848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2626] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:16.765963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.827043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e232f-0ca3-447a-adca-427d25f68f4c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.827722 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b2e18-d097-4ed0-af9a-718942417c7b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.831889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.832621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.832784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.832894 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.833176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.833446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.833729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.833883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.834043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.834680 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.835803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.836581 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.867685 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2627] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:16.867896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2628] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:16.869457 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.869580 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.911902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.932861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89c68-7787-4048-946e-ab7ae45bfc13 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.934274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150442a-750d-4041-affe-7e37009effc5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.937474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.938154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.938438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.938715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.939399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.939519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.940077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.940391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.941225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.941439 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.941934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.943840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.969985 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2629] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:16.970308 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2630] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:16 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:16.970444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.971074 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:16.977887 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:16.978473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e982ad-2950-4422-9b14-c488e05b7150 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.982243 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94e78f-8cde-4e79-9462-8ad5bf028f34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:16.983498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.984208 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.984494 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.984777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.985570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.987250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.989630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:16.990342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:16.990635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:16.990949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:16.991663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:16.993372 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:16.998206 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2631] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:17.002231 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.007646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2632] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:17.008595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.012141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3040e7-bd2a-476d-b284-96029ca295ab admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.016836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.017720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.018002 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.018274 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] PATH_INFO: `/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f/users/39ee8a88c9c84c568dcfb51c3fd10e48/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.018914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cd0b0-a810-4594-bdf1-11524bc3cd23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.024025 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.024934 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.025226 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.025505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] PATH_INFO: `/v3/projects/f0a71f86fc00433aa0559aeb12fe4648/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.033277 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67520ee3c1b4432f8bf49a4d2c86950f, user_id=39ee8a88c9c84c568dcfb51c3fd10e48, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.035957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.041405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a71f86fc00433aa0559aeb12fe4648, user_id=75d5c6e74c1b4bf8b7ae458aa7bd4f53, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.044399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.050226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2633] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:17 2022] PUT /identity/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f/users/39ee8a88c9c84c568dcfb51c3fd10e48/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:17.054230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.059389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2634] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:17 2022] PUT /identity/v3/projects/f0a71f86fc00433aa0559aeb12fe4648/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:17.063002 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.123582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199703e-2de1-4063-b507-ba7b633ecc42 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.129190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d1d2d-2785-4944-b77e-69bf02668dc9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.129478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.130178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.130465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.130743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.131474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.133146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.133860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.134522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.134804 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.135081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.135808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.137497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.145721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2635] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:17.152086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2636] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:17.152844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.153270 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.163647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf141a1-1394-4a1b-9def-41765bf240cf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.164203 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943955b-a51a-42d7-a326-2bbc300ed9e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.169329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.170279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.170473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.170770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] PATH_INFO: `/v3/projects/f0a71f86fc00433aa0559aeb12fe4648/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.170911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.171712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.172066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.172225 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] PATH_INFO: `/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f/users/39ee8a88c9c84c568dcfb51c3fd10e48/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.172889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a71f86fc00433aa0559aeb12fe4648, user_id=75d5c6e74c1b4bf8b7ae458aa7bd4f53, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.174419 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67520ee3c1b4432f8bf49a4d2c86950f, user_id=39ee8a88c9c84c568dcfb51c3fd10e48, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.175598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daa68e-37de-4510-b359-b53f64832107 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.177143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d9b15-091c-458b-b0d1-9c7f3355c5e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.191508 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2637] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:17 2022] PUT /identity/v3/projects/f0a71f86fc00433aa0559aeb12fe4648/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:17.191736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2638] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:17 2022] PUT /identity/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f/users/39ee8a88c9c84c568dcfb51c3fd10e48/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:17.197851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.198135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.198409 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.198628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.198921 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.199200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.228584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] MFA Rules not processed for user `39ee8a88c9c84c568dcfb51c3fd10e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:17.229623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] MFA Rules not processed for user `75d5c6e74c1b4bf8b7ae458aa7bd4f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:17.229800 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:17.230824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:17.308434 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2639] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:17.320223 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2640] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:17.320428 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.321595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:17.326767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:17.327637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.329112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:17.371541 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:17.382438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96700a53-cf9a-47fe-9a6c-6aa623ea5ce1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.386159 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.386836 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.387121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.387393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.387527 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb210264-7ca7-4583-b60a-f5645670bb52 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.391156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.391802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.392081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.392357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.396599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.398045 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.401398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.402817 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.407360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2641] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:17.412617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2642] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:20:17.480233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.482016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:17.489997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ea5b-53f4-4ade-9985-9e74edad4d9e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.490788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca346-75e1-4b77-b772-23b78211de69 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:17.493934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.495012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.495143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:17.495316 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.495565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.495666 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:17.495945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:17.496220 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:17.553093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.553564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:17.554561 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.555017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34323f4-1ebf-48f1-9c74-bc2def9d9293 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:17.563940 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2643] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:20:17.564648 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2644] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:27.559816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=37 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:27.560760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=53 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:32.751393 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:32.821738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6ad90-7a48-42aa-b5b6-5103d1b4c1e9 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:32.826612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:32.828184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:32.828184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:32.828184 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] PATH_INFO: `/v3/users/f35848201c8c4c3b8f0d71d5e1ade7c7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:32.829416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35848201c8c4c3b8f0d71d5e1ade7c7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:32.831778 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:32.885553 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35848201c8c4c3b8f0d71d5e1ade7c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:32.886212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35848201c8c4c3b8f0d71d5e1ade7c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:32.886807 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35848201c8c4c3b8f0d71d5e1ade7c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:32.900273 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35848201c8c4c3b8f0d71d5e1ade7c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:32.910314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35848201c8c4c3b8f0d71d5e1ade7c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:32.922592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2645] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:32 2022] DELETE /identity/v3/users/f35848201c8c4c3b8f0d71d5e1ade7c7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:32.927370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:32.927859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:32.927957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:32.946793 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:32.947872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.016196 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2646] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:33.030593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.032629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.094336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b8e01-22de-4aba-8865-d598c2ec21b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:33.099004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:33.099842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.100127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.100400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.112281 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:33.113777 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:33.125320 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2647] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:33.413517 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.415076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.427656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770af438-0eae-42fb-a842-00655e8efe06 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:33.435142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:33.435408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.435950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.436073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] PATH_INFO: `/v3/projects/5d56e45a575b4af58dd61c673aeaaf45` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.438191 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d56e45a575b4af58dd61c673aeaaf45)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:33.440550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:33.492848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d56e45a575b4af58dd61c673aeaaf45', 'request_id': 'req-7dbbac23-b153-4312-bc94-1c07be95fee8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.498804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d56e45a575b4af58dd61c673aeaaf45', 'request_id': 'req-7dbbac23-b153-4312-bc94-1c07be95fee8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.505604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d56e45a575b4af58dd61c673aeaaf45', 'request_id': 'req-7dbbac23-b153-4312-bc94-1c07be95fee8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.522334 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] The token cache is being invalidate because project 5d56e45a575b4af58dd61c673aeaaf45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:33.522617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.523379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.525080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2648] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:33 2022] DELETE /identity/v3/projects/5d56e45a575b4af58dd61c673aeaaf45 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:33.529363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.544053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48010e-5632-4327-a6b5-a4e57cd845ef service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:33.564563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:33.597173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.598666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.605396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:33.606400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.606688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.606962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] PATH_INFO: `/v3/users/39ee8a88c9c84c568dcfb51c3fd10e48` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.608179 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ee8a88c9c84c568dcfb51c3fd10e48)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:33.610133 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:33.657593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ee8a88c9c84c568dcfb51c3fd10e48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.658247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ee8a88c9c84c568dcfb51c3fd10e48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.658857 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ee8a88c9c84c568dcfb51c3fd10e48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.665953 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbac23-b153-4312-bc94-1c07be95fee8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:33.670791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:33.670941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ee8a88c9c84c568dcfb51c3fd10e48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.671725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.672011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.672281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] PATH_INFO: `/v3/users/4a563644bdf946038482e54b61c6708b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.673585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a563644bdf946038482e54b61c6708b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:33.675891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:33.679567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35086fc6-6914-4205-bd08-02d36ec079a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ee8a88c9c84c568dcfb51c3fd10e48'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.691828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2649] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:33 2022] DELETE /identity/v3/users/39ee8a88c9c84c568dcfb51c3fd10e48 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:33.705738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:33.726084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a563644bdf946038482e54b61c6708b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.726774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a563644bdf946038482e54b61c6708b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.739994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a563644bdf946038482e54b61c6708b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.740818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a563644bdf946038482e54b61c6708b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.748792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a563644bdf946038482e54b61c6708b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:33.763113 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2650] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:33 2022] DELETE /identity/v3/users/4a563644bdf946038482e54b61c6708b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:33.767810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.768352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.768473 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.785649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:33.786860 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.859490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2651] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:33.869653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.871185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:33.933971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1531a-d43a-4bbd-a7a0-9de366a154b4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:33.937806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:33.938471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:33.938757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:33.939031 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:33.949933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:33.951689 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562b39c-0ba1-4f96-9d64-ac5bc7dbd170 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:33.965762 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2652] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:33.988124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:33.989938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.006855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b4693-b0b7-4adc-9cf7-5574c53ccdc4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.013599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.014655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.014939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.015511 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] PATH_INFO: `/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.016608 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67520ee3c1b4432f8bf49a4d2c86950f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.018693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.071685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:34.089332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67520ee3c1b4432f8bf49a4d2c86950f', 'request_id': 'req-a41c07f7-111d-4ea4-b353-08319dcab66d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.098134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67520ee3c1b4432f8bf49a4d2c86950f', 'request_id': 'req-a41c07f7-111d-4ea4-b353-08319dcab66d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.108219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67520ee3c1b4432f8bf49a4d2c86950f', 'request_id': 'req-a41c07f7-111d-4ea4-b353-08319dcab66d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.132733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] The token cache is being invalidate because project 67520ee3c1b4432f8bf49a4d2c86950f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:34.132733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.132733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a41c07f7-111d-4ea4-b353-08319dcab66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.132733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2653] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:33 2022] DELETE /identity/v3/projects/67520ee3c1b4432f8bf49a4d2c86950f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:34.134855 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.135154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.135441 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.180637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.181945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.257888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2654] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:34.266405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.266701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.267006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.284982 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.286896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.289700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.291257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.302399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d1ca0-293d-49e0-bb4f-b4bd34e032b0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.304181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2655] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:34.308577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.312017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.314019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.314464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.315121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] PATH_INFO: `/v3/projects/19f2e5a7cdbe418189c25c72448252ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.316809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f2e5a7cdbe418189c25c72448252ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.319536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.319835 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aa9f8-3c83-4908-9802-6f8408d1df90 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.325611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.326348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.326704 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.327152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.327951 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:34.328329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.330031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.343384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2656] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:20:34.347865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.358761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4632f-70f0-48b1-b9d6-60100f8321ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.364476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:34.373683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f2e5a7cdbe418189c25c72448252ab', 'request_id': 'req-7588e749-69e6-408a-af56-069ab975e7be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.380819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f2e5a7cdbe418189c25c72448252ab', 'request_id': 'req-7588e749-69e6-408a-af56-069ab975e7be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.387302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f2e5a7cdbe418189c25c72448252ab', 'request_id': 'req-7588e749-69e6-408a-af56-069ab975e7be'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.400558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] The token cache is being invalidate because project 19f2e5a7cdbe418189c25c72448252ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:34.400851 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.401489 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7588e749-69e6-408a-af56-069ab975e7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:34.403057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2657] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:34 2022] DELETE /identity/v3/projects/19f2e5a7cdbe418189c25c72448252ab => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:34.407988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d34d-3eb8-47a0-891e-264e65d079b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.408361 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d34d-3eb8-47a0-891e-264e65d079b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.408862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d34d-3eb8-47a0-891e-264e65d079b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.419587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.420254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.420576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.420853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.421543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.423450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.432597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f4a2d34d-3eb8-47a0-891e-264e65d079b9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.433820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f4a2d34d-3eb8-47a0-891e-264e65d079b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.443211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2658] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:34.450779 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.451071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.451255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2659] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:34.451362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.455104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.456848 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.469775 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.471033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.520138 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0bcf4-7614-4f4a-8302-e260a692993d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.525375 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.526057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.526349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.526625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.527329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.529294 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.535694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2660] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:34.540463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.549773 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1cd4e-c699-439c-8960-d27eff4a2c00 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.554141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2661] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:34.554633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.554977 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.555292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.555573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.555845 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.556610 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:34.556917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.558555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.565832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211ed93-a851-45ce-966c-5a0a6f6a2283 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.570626 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.574597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2662] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:20:34.574768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.574768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.574768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.574768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.574768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.575565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.588519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1941a-da70-4a82-914c-c0b33875b158 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.588703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2663] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:34.590086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.590362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.591054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.591333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.591606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.592315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.594235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.600031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5965d-a74d-4dba-8a47-42b885762e3a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.604823 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.605474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.605851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.606026 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.606746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.608336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.610732 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:34.623011 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2664] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:34.626010 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2665] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:34.626289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.626956 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.638242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6045e3e-95df-435f-9fcf-97c29dfbd00d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.692558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f8de9-d763-4840-9aea-97cbb6e8b443 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.696571 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.697479 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.697596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.697770 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.698053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] PATH_INFO: `/v3/projects/18871c6738b6440daa4e081562312cd9/users/b5f15b49b4a8416fb1f67bf95a8ff859/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.698246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.698444 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.698721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.699418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.701402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.714095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18871c6738b6440daa4e081562312cd9, user_id=b5f15b49b4a8416fb1f67bf95a8ff859, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.717566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa03c94-66d2-44f9-8396-699e16f88a29 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.724876 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2666] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:34.728797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.735195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2667] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:34 2022] PUT /identity/v3/projects/18871c6738b6440daa4e081562312cd9/users/b5f15b49b4a8416fb1f67bf95a8ff859/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:34.739339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9c80e-5b41-45f3-b259-9731ca7e5cbc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.747432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.747730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.748014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.754146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:34.777719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] MFA Rules not processed for user `b5f15b49b4a8416fb1f67bf95a8ff859`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.778852 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.805614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.806309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.806572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.806840 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.807565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.809296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.819198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2668] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:34.823086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.833114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0be13c-a8d4-4bfe-a3e7-4f09fae61677 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.837820 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.838468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.838744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.839019 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.839738 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.841372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.847062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2669] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:34.851108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.852328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2670] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:34.856149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.862686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58969b-3854-415e-9d44-1e522868dfab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.866021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2e484-3423-47d8-a71a-d54cd1d3c76a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:34.867085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.867869 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.868144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.868433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.869465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.871144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:34.871332 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.872014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.872299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.872608 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] PATH_INFO: `/v3/projects/c37acb9f83ab4adaa0d7f06cf31394b3/users/7414189a96fa438ab03ebdcc2914b5fe/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.887782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37acb9f83ab4adaa0d7f06cf31394b3, user_id=7414189a96fa438ab03ebdcc2914b5fe, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:34.890740 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c9b55-5d96-481c-88ad-5b991521f7b9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:34.898170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2671] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:34.902498 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:34.904602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2672] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:34 2022] PUT /identity/v3/projects/c37acb9f83ab4adaa0d7f06cf31394b3/users/7414189a96fa438ab03ebdcc2914b5fe/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:34.915247 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:34.915538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:34.915918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:34.942673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] MFA Rules not processed for user `7414189a96fa438ab03ebdcc2914b5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:34.943892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:34.953515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:34.969010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5f412-ba42-4f6e-a1f6-98ba25239d36 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.009134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2673] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:35.020587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.022054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:35.031437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.032055 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.032347 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.032660 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.033361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.035466 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.058041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2674] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:34 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:35.061939 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.073188 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27b376-a964-42be-b785-4aa573cfa2b7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.078642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.079302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.079591 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.079870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.080738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.082343 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.083593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8212c8-c615-4395-b63b-c0ac76c61541 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.087351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.088003 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.088288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.088657 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.092621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2675] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:35.096767 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.099548 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.101049 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.106094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c5f3-a82f-451c-a55f-f5198ea747e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.110604 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2676] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:20:35.110730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.111745 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.112036 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.112306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] PATH_INFO: `/v3/projects/4af4b12745754bdbb159b4f5d05ebd39/users/90a24a82f16942fdb301ae68ea8d1bb8/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.127619 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af4b12745754bdbb159b4f5d05ebd39, user_id=90a24a82f16942fdb301ae68ea8d1bb8, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.130292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.145107 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2677] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:35 2022] PUT /identity/v3/projects/4af4b12745754bdbb159b4f5d05ebd39/users/90a24a82f16942fdb301ae68ea8d1bb8/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:35.149284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.161566 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740b800-f49a-43e7-ac83-5f5338809281 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.170824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:35.225202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.226872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.227233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.227582 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.228881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.230629 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9910c-e879-4978-9b41-09e78a2dea6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.245663 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2678] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:35.249794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.263251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f19110-5cf7-4ea4-bdfd-8d73a44a3d17 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.271174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.272092 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.272813 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.273097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] PATH_INFO: `/v3/projects/4af4b12745754bdbb159b4f5d05ebd39/users/90a24a82f16942fdb301ae68ea8d1bb8/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.276233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af4b12745754bdbb159b4f5d05ebd39, user_id=90a24a82f16942fdb301ae68ea8d1bb8, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.279102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.292491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2679] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:35 2022] PUT /identity/v3/projects/4af4b12745754bdbb159b4f5d05ebd39/users/90a24a82f16942fdb301ae68ea8d1bb8/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:35.304829 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.305116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.305440 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.339949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] MFA Rules not processed for user `90a24a82f16942fdb301ae68ea8d1bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:35.342018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:35.435515 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2680] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:35.451236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.514421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24624eb-c2cf-4bfd-b279-e57c2a94259c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.518428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.519112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.519353 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.519639 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.534352 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.535729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.549895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2681] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:35.644529 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:35.654763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad68e9f-9de9-4f7e-a04f-d2d8a2074109 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:35.659009 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:35.659812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:35.660124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:35.660404 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:35.719600 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:35.721106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b0314-0444-469a-a0d6-54da2151d0c4 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:35.730200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2682] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:36.018559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:36.063231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:36.086082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cc21c-6498-45cb-85b7-126131d47f7a service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:36.092856 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:36.094336 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:36.094812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:36.095192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] PATH_INFO: `/v3/users/b5f15b49b4a8416fb1f67bf95a8ff859` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:36.097096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f15b49b4a8416fb1f67bf95a8ff859)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:36.099024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:36.148916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f15b49b4a8416fb1f67bf95a8ff859'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.149577 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f15b49b4a8416fb1f67bf95a8ff859'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.150194 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f15b49b4a8416fb1f67bf95a8ff859'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.163153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f15b49b4a8416fb1f67bf95a8ff859'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.173240 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f15b49b4a8416fb1f67bf95a8ff859'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.185678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2683] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:36 2022] DELETE /identity/v3/users/b5f15b49b4a8416fb1f67bf95a8ff859 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:36.190826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:36.191333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:36.191333 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:36.208390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:36.209625 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:36.237713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:36.272542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2684] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:36.283649 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:36.285104 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:36.344441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd02144-cb97-42aa-9d67-ef64f138a318 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:36.349174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:36.349843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:36.350159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:36.350435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:36.360772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:36.362590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:36.372945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:36.683239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:36.684718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:36.697124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789549a-527e-4cde-8d71-e6811331adc9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:36.702146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:36.703011 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:36.703294 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:36.703576 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] PATH_INFO: `/v3/projects/18871c6738b6440daa4e081562312cd9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:36.704831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18871c6738b6440daa4e081562312cd9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:36.706850 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:36.768154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18871c6738b6440daa4e081562312cd9', 'request_id': 'req-27d6d563-b562-49bd-8f8e-6b2c359c7d21'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.776372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18871c6738b6440daa4e081562312cd9', 'request_id': 'req-27d6d563-b562-49bd-8f8e-6b2c359c7d21'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.783806 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18871c6738b6440daa4e081562312cd9', 'request_id': 'req-27d6d563-b562-49bd-8f8e-6b2c359c7d21'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.799160 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] The token cache is being invalidate because project 18871c6738b6440daa4e081562312cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:36.799623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.800416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.805501 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2686] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:36 2022] DELETE /identity/v3/projects/18871c6738b6440daa4e081562312cd9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:36.806311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:36.817874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0dc84-64fa-4030-b3fd-b4325bd886b0 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:36.882938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:36.884369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:36.884705 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:36.884986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] PATH_INFO: `/v3/users/90a24a82f16942fdb301ae68ea8d1bb8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:36.886177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a24a82f16942fdb301ae68ea8d1bb8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:36.888178 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:36.890625 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:36.940762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a24a82f16942fdb301ae68ea8d1bb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.941432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a24a82f16942fdb301ae68ea8d1bb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.942056 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a24a82f16942fdb301ae68ea8d1bb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.954043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a24a82f16942fdb301ae68ea8d1bb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.962820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b4f83c8-50b5-40f9-94a8-6d781154e293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a24a82f16942fdb301ae68ea8d1bb8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:36.975968 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2687] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:36 2022] DELETE /identity/v3/users/90a24a82f16942fdb301ae68ea8d1bb8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:37.257455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.258971 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.272606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6d563-b562-49bd-8f8e-6b2c359c7d21 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.334716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.335587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.335867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.336140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] PATH_INFO: `/v3/projects/4af4b12745754bdbb159b4f5d05ebd39` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.337402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af4b12745754bdbb159b4f5d05ebd39)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.339324 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.395550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af4b12745754bdbb159b4f5d05ebd39', 'request_id': 'req-13fe39ef-5913-4b28-834b-b0289a018f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.400154 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:37.401645 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af4b12745754bdbb159b4f5d05ebd39', 'request_id': 'req-13fe39ef-5913-4b28-834b-b0289a018f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.410995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af4b12745754bdbb159b4f5d05ebd39', 'request_id': 'req-13fe39ef-5913-4b28-834b-b0289a018f76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.428012 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] The token cache is being invalidate because project 4af4b12745754bdbb159b4f5d05ebd39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:37.428314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.428936 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13fe39ef-5913-4b28-834b-b0289a018f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.430519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2688] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:37 2022] DELETE /identity/v3/projects/4af4b12745754bdbb159b4f5d05ebd39 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:37.437757 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.438288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.438437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.457061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:37.458288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.538057 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2689] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:37.546771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.547064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.547447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.572831 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:37.574075 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.583498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2690] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:37.588143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.599155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e423da4-589e-46e2-9882-136b639d28cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.605718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.606396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.606708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.606995 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.607737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:37.608050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.609728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.624674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2691] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:20:37.629383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.639493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a65b-14ea-4396-87c7-0371c759c6f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.647327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.648058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.648341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.648656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.649389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.651549 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.688550 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2692] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:37.690623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.703125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2cf5b-05bc-4e45-9f21-13ea310b6657 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.709329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.711285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.774121 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.774797 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.775080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.775428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.776124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.778213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.792174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5cab15-5e60-4def-bc02-bc1ee5c341d3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.799031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:37.807633 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.808174 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2693] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:37.808846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.809369 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.809688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] PATH_INFO: `/v3/users/7414189a96fa438ab03ebdcc2914b5fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.811666 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7414189a96fa438ab03ebdcc2914b5fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.813146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.814004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.823238 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de473-cc06-4975-937a-7f595e0b8df7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.828508 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.829399 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.829866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.830568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.831409 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.833834 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.850326 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2694] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:20:37.853351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.865471 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250810e1-fd95-46da-a8d5-a7389a56736e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.870771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.872382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.872980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.873272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.873843 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7414189a96fa438ab03ebdcc2914b5fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.874102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.874560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7414189a96fa438ab03ebdcc2914b5fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.876157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.894314 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2695] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:37.894777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7414189a96fa438ab03ebdcc2914b5fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.896871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7414189a96fa438ab03ebdcc2914b5fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.898341 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:37.901125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.908376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68bc331e-049b-4ed2-a13a-6533822e9a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7414189a96fa438ab03ebdcc2914b5fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:37.915349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f80de-4cf2-4bce-b0cf-d4ee6c4367a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:37.922674 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:37.923587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.923884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.924392 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] PATH_INFO: `/v3/projects/5e57658c9c9340adba6ab15050b24e8a/users/994fd8a1975c45a8a65ace3d090d1ff5/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.929309 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2696] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:20:37 2022] DELETE /identity/v3/users/7414189a96fa438ab03ebdcc2914b5fe => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:37.933802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.934818 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.935386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.942245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e57658c9c9340adba6ab15050b24e8a, user_id=994fd8a1975c45a8a65ace3d090d1ff5, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:37.945047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c881ce0-4d5c-44e4-a180-3427df78ef5e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:37.958245 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:37.959902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.961362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2697] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:37 2022] PUT /identity/v3/projects/5e57658c9c9340adba6ab15050b24e8a/users/994fd8a1975c45a8a65ace3d090d1ff5/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:37.966876 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:37.967159 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:37.967442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:37.986082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:37.987717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:37.997617 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:38.034952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2698] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:38.049073 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:38.050757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:38.056946 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2699] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:38.067820 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:38.070094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:38.130487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa914c-4854-4272-bb50-4078529f3949 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:38.135425 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:38.136285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.136601 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.136867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.144697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492748b-dec6-4648-9dc1-f94c0c6f2171 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:38.148907 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:38.149270 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.149617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.149958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.153727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:38.155485 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:38.165793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:38.169432 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d66691-20c2-4546-89a7-4936fbdeca2a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:38.170474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2700] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:20:38.184007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2701] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:38.668086 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:38.670448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:38.695745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79b6c5-a534-4f96-ae68-e6f7f2485cfe service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:38.700731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:38.701885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.702173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.702448 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] PATH_INFO: `/v3/projects/c37acb9f83ab4adaa0d7f06cf31394b3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.703980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37acb9f83ab4adaa0d7f06cf31394b3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:38.706078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:38.756636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37acb9f83ab4adaa0d7f06cf31394b3', 'request_id': 'req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:38.763042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37acb9f83ab4adaa0d7f06cf31394b3', 'request_id': 'req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:38.769920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37acb9f83ab4adaa0d7f06cf31394b3', 'request_id': 'req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:38.783921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] The token cache is being invalidate because project c37acb9f83ab4adaa0d7f06cf31394b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:20:38.784223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:38.784911 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7c232ee0-96a3-487b-a7ae-6f9dd35b9ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:20:38.786474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2702] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:20:38 2022] DELETE /identity/v3/projects/c37acb9f83ab4adaa0d7f06cf31394b3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:38.791364 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.791868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.791971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.813981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:38.815443 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:38.870501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:38.883699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2703] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:38.892660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.893155 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.893531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.915620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:38.917059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:38.927981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2704] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:38.931954 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:38.942454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594f6ed-d8ae-46f5-8ee2-a5cb622c98fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:38.947385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:38.948285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.948458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.948634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.949415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:20:38.949743 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:38.951415 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:38.965369 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2705] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:20:38.970873 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:38.981915 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61971cd-d139-4f3c-8782-f05e2519664d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:38.986917 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:38.988563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:38.988563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:38.988563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:38.988997 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:38.990930 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.023317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2706] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:38 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:39.027366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:39.037864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c1f0-357f-4a83-92cb-659bf6a0f59f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:39.103015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:39.103698 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.103979 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.104255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.104996 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:39.106936 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.135681 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2707] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:39 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:20:39.139682 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:39.154598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2e4-13a9-4b6f-a745-9d9a46125b16 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:39.165568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:39.166658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.167973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.168510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.169506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:39.171522 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.187510 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2708] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:39.188784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:39.199080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cef9f-e1b4-4274-9c22-1cb102240af0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:39.203927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:39.204693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.205045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.205420 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.206210 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:39.207941 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd2d01-5461-423c-b170-3558a4225383 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.221768 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2709] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:39.223360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:39.234337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b732e6-965e-4e6f-9985-c088b3e6ae1d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:39.239759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:39.240468 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.240824 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.241411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] PATH_INFO: `/v3/projects/775518eb8c104022b5acfd12ead0fee5/users/c3c55aee07d4447489ddedde5972ec58/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.258593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775518eb8c104022b5acfd12ead0fee5, user_id=c3c55aee07d4447489ddedde5972ec58, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:39.261354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.271545 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:39.282071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2710] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:39 2022] PUT /identity/v3/projects/775518eb8c104022b5acfd12ead0fee5/users/c3c55aee07d4447489ddedde5972ec58/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:39.284176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0869-5d86-423b-9cb6-e81710f91c3c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.285122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0869-5d86-423b-9cb6-e81710f91c3c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.285809 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e0869-5d86-423b-9cb6-e81710f91c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.305677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f0e0869-5d86-423b-9cb6-e81710f91c3c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:39.306990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f0e0869-5d86-423b-9cb6-e81710f91c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:39.377418 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2711] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:39.387239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:39.389032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:39.457465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891565-20f6-4f01-aae8-331b8bfeb033 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:39.463703 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:39.464558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:39.464920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:39.465376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:39.477774 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:39.479332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:39.490631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2712] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:44.477808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:44.478339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:44.478339 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:44.544755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] MFA Rules not processed for user `994fd8a1975c45a8a65ace3d090d1ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:44.546163 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:44.632931 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:44.696548 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2713] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:44.698631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:44.700655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:44.724340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad569-707c-4d7b-b3e0-9e1d5bd57879 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:44.732377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:44.733104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:44.733464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:44.733797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:44.734541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:44.737385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:44.791867 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2714] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:44 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:44.793438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:44.812097 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb997e5-7341-4d26-9832-1b9c9abfd2bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:44.909117 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:44.909963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:44.910398 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:44.910772 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:44.916318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:44.919082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:44.951461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2715] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:44 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:44.955621 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:44.972195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822cf8-9036-41ea-bcdb-7be786bfdb06 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:44.985574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:44.986929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:44.987708 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:44.989027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:44.990950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:44.993080 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:45.017392 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2716] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:45.023652 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:45.038388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c01b2-35da-40fd-a30c-9d6052c83e9e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:45.047422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:45.056264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:45.056912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:45.058081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] PATH_INFO: `/v3/projects/e182160965d24221bff6069a0b1a89a3/users/00d3c70fd31144018c3fc93e893fa6f3/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:45.083183 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e182160965d24221bff6069a0b1a89a3, user_id=00d3c70fd31144018c3fc93e893fa6f3, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:45.086784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:45.105492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2717] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:45 2022] PUT /identity/v3/projects/e182160965d24221bff6069a0b1a89a3/users/00d3c70fd31144018c3fc93e893fa6f3/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:45.109590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:45.122144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2bd22-51f1-4213-9b4f-7c655f5c5819 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:45.201178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:45.201890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:45.202181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:45.202462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:45.203254 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:45.205077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25edbb7-87ff-4485-a641-4f050b0ebb9e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:45.219337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2718] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:45.223384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:45.234170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16380607-c19d-46d0-ae20-d324e035ef95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:45.239487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:45.240401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:45.240723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:45.241001 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] PATH_INFO: `/v3/projects/e182160965d24221bff6069a0b1a89a3/users/00d3c70fd31144018c3fc93e893fa6f3/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:45.243407 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e182160965d24221bff6069a0b1a89a3, user_id=00d3c70fd31144018c3fc93e893fa6f3, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:45.246714 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:45.259806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2719] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:45 2022] PUT /identity/v3/projects/e182160965d24221bff6069a0b1a89a3/users/00d3c70fd31144018c3fc93e893fa6f3/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:45.994513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:45.996881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:45.997387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:46.060587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] MFA Rules not processed for user `c3c55aee07d4447489ddedde5972ec58`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:46.063323 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:46.207450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:46.253990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2720] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:46.258855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:46.462992 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864e9cf-b6b0-4787-b08c-e43bc62f4330 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:46.491631 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:46.492468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:46.493110 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:46.494895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:46.501962 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:46.507144 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:46.574454 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2721] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:20:46 2022] POST /identity/v3/projects => generated 373 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:46.578626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:46.602597 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3fd8c-0c20-4a3a-b276-ebcf2393ff6d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:46.762912 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:46.763727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:46.764059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:46.764337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:46.765314 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:46.767715 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:46.805456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2722] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:20:46 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:20:46.807292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:46.824504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab680ff4-48e8-480b-b2c6-3b8c4fd185b5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:46.841464 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:46.842180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:46.842505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:46.842788 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:46.843924 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:46.851539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:46.872831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2723] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:46.878142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:46.902416 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c5bd0-260e-40ae-90af-9e6880f4fd6d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:46.918760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:46.919651 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:46.919933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:46.920213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] PATH_INFO: `/v3/projects/9fe0fa323179498db3572def1af95429/users/cd337799adad492d93eb07e245a42387/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:46.963614 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe0fa323179498db3572def1af95429, user_id=cd337799adad492d93eb07e245a42387, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:46.968345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:46.998888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2724] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:46 2022] PUT /identity/v3/projects/9fe0fa323179498db3572def1af95429/users/cd337799adad492d93eb07e245a42387/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:20:47.003214 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:47.017442 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da38d-4a4d-401e-845e-72a3ac710cdd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:47.048017 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:47.128251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:47.129006 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:47.129299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:47.129584 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:47.130888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:47.132730 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be15b2-766e-495e-b9fc-7e4b3c1a2f8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:47.156605 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2725] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:20:47.157565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:47.172302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84caca-06f5-446c-a12f-13ac61a4a4cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:47.187027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:47.187961 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:47.188251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:47.188562 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] PATH_INFO: `/v3/projects/9fe0fa323179498db3572def1af95429/users/cd337799adad492d93eb07e245a42387/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:47.193478 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe0fa323179498db3572def1af95429, user_id=cd337799adad492d93eb07e245a42387, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:47.196821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:47.219185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2726] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:20:47 2022] PUT /identity/v3/projects/9fe0fa323179498db3572def1af95429/users/cd337799adad492d93eb07e245a42387/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:20:50.869793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b43f-85f5-4a9d-832d-fde0f410b79c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:50.870435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b43f-85f5-4a9d-832d-fde0f410b79c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:50.870435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6b43f-85f5-4a9d-832d-fde0f410b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:50.919031 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83e6b43f-85f5-4a9d-832d-fde0f410b79c None None] MFA Rules not processed for user `00d3c70fd31144018c3fc93e893fa6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:50.920461 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83e6b43f-85f5-4a9d-832d-fde0f410b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:51.040181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2727] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:51.053570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:51.056329 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:51.199903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f2fe-d099-44f7-b275-6ab9d9bd237c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:51.203896 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c27-7597-4293-8805-5066dca106da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bdf485ec3ca4f3785bb47f3706c4466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:51.204637 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c27-7597-4293-8805-5066dca106da service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:51.204934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c27-7597-4293-8805-5066dca106da service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:51.205664 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec70c27-7597-4293-8805-5066dca106da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:51.372399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec70c27-7597-4293-8805-5066dca106da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:51.377709 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec70c27-7597-4293-8805-5066dca106da service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:51.393921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:51.401293 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2728] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:20:51.609560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2bd-6743-48f9-b71f-dc48ca694c2f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:51.609853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2bd-6743-48f9-b71f-dc48ca694c2f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:51.610180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2bd-6743-48f9-b71f-dc48ca694c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:51.647306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c26e2bd-6743-48f9-b71f-dc48ca694c2f None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:51.653207 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c26e2bd-6743-48f9-b71f-dc48ca694c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:51.811900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2729] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:20:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:51.913706 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c27-7597-4293-8805-5066dca106da service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:51.915276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aec70c27-7597-4293-8805-5066dca106da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:52.066199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec70c27-7597-4293-8805-5066dca106da service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:52.080881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:52.081581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:52.081883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:52.082137 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:52.097849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:52.100389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:52.114028 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2730] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:20:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:20:53.021129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753778c-4d6c-4efb-8b5a-a9b89d351534 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:53.025384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753778c-4d6c-4efb-8b5a-a9b89d351534 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:53.025913 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753778c-4d6c-4efb-8b5a-a9b89d351534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:53.060959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9753778c-4d6c-4efb-8b5a-a9b89d351534 None None] MFA Rules not processed for user `cd337799adad492d93eb07e245a42387`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:53.062588 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9753778c-4d6c-4efb-8b5a-a9b89d351534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:53.111560 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:53.166060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2731] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:20:53.187869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:53.189491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:53.352064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1f865-fa6d-4ef9-a906-5cbbd6ce10b5 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:53.372590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:53.373634 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:53.374133 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:53.374622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:53.493684 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:53.495862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:53.511628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2732] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:20:55.285317 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93c1c0-7982-43e3-b160-979fa3ef57cb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:55.285833 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93c1c0-7982-43e3-b160-979fa3ef57cb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:55.285969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93c1c0-7982-43e3-b160-979fa3ef57cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:55.316621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f93c1c0-7982-43e3-b160-979fa3ef57cb None None] MFA Rules not processed for user `c3c55aee07d4447489ddedde5972ec58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:20:55.317895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f93c1c0-7982-43e3-b160-979fa3ef57cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:55.337798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2733] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:20:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:20:55.833435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:20:55.920398 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:20:55.968250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2798a-72ce-42ff-9b52-2160ef2c5c72 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:20:55.972584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:20:55.973475 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:20:55.973954 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:20:55.974534 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:20:55.977211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:20:55.996200 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:20:55.999056 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:20:56.012890 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:20:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:04.622838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:04.623794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:04.624082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:04.645262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] MFA Rules not processed for user `994fd8a1975c45a8a65ace3d090d1ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:04.646486 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:04.654868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2735] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:04.943817 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:04.954318 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b010ad-65d2-4c07-908b-b6f1363ba18e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:04.959227 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:04.959900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:04.960189 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:04.960513 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:04.962616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:04.975949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:04.977479 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600601c9-127c-422b-ba6a-fcd1bbd8c3db service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:04.995289 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:05.338692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:05.340135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:05.415304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5243d-00bd-4850-8891-5c4e2981f5b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:05.420377 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:05.421082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:05.421363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:05.421646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:05.423645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:05.436484 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:05.438093 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:05.448277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:12.920568 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:12.921135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:12.921135 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:12.940585 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:12.941984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:12.949616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2738] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:21:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:12.961736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:12.973373 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90e056-4d7c-40ef-b89c-7a4364ca98ca service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:12.978501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:12.979891 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:12.980176 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:12.980487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:12.988884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:12.990496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be1105-8e1d-47a8-a456-ddce084975bd service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:12.998905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2739] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:21:16.434464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:16.436006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:16.526504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76677d67-5c34-4517-a29d-3c87b0d0badc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:16.531068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:16.532083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:16.532372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:16.532714 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] PATH_INFO: `/v3/users/16929edc079c45bb90de1c6b576abb24` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:16.533937 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16929edc079c45bb90de1c6b576abb24)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:16.535893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:16.594975 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16929edc079c45bb90de1c6b576abb24'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:16.595720 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16929edc079c45bb90de1c6b576abb24'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:16.609526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16929edc079c45bb90de1c6b576abb24'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:16.610431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16929edc079c45bb90de1c6b576abb24'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:16.622670 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16929edc079c45bb90de1c6b576abb24'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:16.653350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2740] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:16 2022] DELETE /identity/v3/users/16929edc079c45bb90de1c6b576abb24 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:16.658696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:16.659703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:16.659998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:16.679221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:16.680700 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:16.748972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2741] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:16.759661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:16.761134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:16.795174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:16.827955 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8882a34-ed46-4e39-bf54-e5dea9cec964 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:16.833402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:16.834279 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:16.834603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:16.835048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:16.853317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:16.855115 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:16.878669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2742] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:17.266426 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:17.267904 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:17.323903 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c3828-c3e2-410a-8739-d6ef241de01f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:17.329617 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:17.330539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:17.330828 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:17.331108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] PATH_INFO: `/v3/projects/d2c9067b02ee4ad988b7a85806a7fdd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:17.332594 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c9067b02ee4ad988b7a85806a7fdd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:17.334587 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:17.338062 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:17.391993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c9067b02ee4ad988b7a85806a7fdd2', 'request_id': 'req-139371e6-63f1-4546-a905-75ee7497acf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.397874 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c9067b02ee4ad988b7a85806a7fdd2', 'request_id': 'req-139371e6-63f1-4546-a905-75ee7497acf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.408066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c9067b02ee4ad988b7a85806a7fdd2', 'request_id': 'req-139371e6-63f1-4546-a905-75ee7497acf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.419868 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] The token cache is being invalidate because project d2c9067b02ee4ad988b7a85806a7fdd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:17.420190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.420870 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.422545 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2743] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:17 2022] DELETE /identity/v3/projects/d2c9067b02ee4ad988b7a85806a7fdd2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:17.426882 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:17.453650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e04396-2af7-43de-b714-a2a508888190 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:17.516977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:17.517999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:17.518292 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:17.518590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] PATH_INFO: `/v3/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:17.519903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d5c6e74c1b4bf8b7ae458aa7bd4f53)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:17.521943 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:17.584502 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d5c6e74c1b4bf8b7ae458aa7bd4f53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.585553 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d5c6e74c1b4bf8b7ae458aa7bd4f53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.600805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d5c6e74c1b4bf8b7ae458aa7bd4f53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.601538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d5c6e74c1b4bf8b7ae458aa7bd4f53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.608506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1dcd59-d945-43c7-b6a0-c2ca4894feb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d5c6e74c1b4bf8b7ae458aa7bd4f53'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:17.625763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2744] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:17 2022] DELETE /identity/v3/users/75d5c6e74c1b4bf8b7ae458aa7bd4f53 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:17.998434 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.000065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:18.084147 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139371e6-63f1-4546-a905-75ee7497acf2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.226438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.227428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.227635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.227911 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] PATH_INFO: `/v3/projects/f0a71f86fc00433aa0559aeb12fe4648` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.229231 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a71f86fc00433aa0559aeb12fe4648)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.231122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.290879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a71f86fc00433aa0559aeb12fe4648', 'request_id': 'req-d7271d49-64e4-443e-abac-3960fc5932aa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:18.295983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:18.298127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a71f86fc00433aa0559aeb12fe4648', 'request_id': 'req-d7271d49-64e4-443e-abac-3960fc5932aa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:18.308845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a71f86fc00433aa0559aeb12fe4648', 'request_id': 'req-d7271d49-64e4-443e-abac-3960fc5932aa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:18.323322 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] The token cache is being invalidate because project f0a71f86fc00433aa0559aeb12fe4648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:18.323615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:18.324272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7271d49-64e4-443e-abac-3960fc5932aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:18.326134 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2745] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:17 2022] DELETE /identity/v3/projects/f0a71f86fc00433aa0559aeb12fe4648 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:18.330969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.331531 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.331976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.353204 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:18.354536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:18.389847 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:18.436078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2746] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:18.437342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.437636 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.437919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.458916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:18.460301 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:18.468558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2747] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:18.473048 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.483813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf8b62-26e1-4246-98e9-77f0300b2a51 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.488701 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.489449 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.489794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.490185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.490994 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:18.491526 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.493574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.506620 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2748] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:18.513524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.523633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981510a-d87c-401b-ab94-455fc4a445e6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.528293 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.528999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.529289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.529567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.530283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.532368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.564385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2749] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:18.565889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.580185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c44118-c8f3-4661-b8f9-3ad5cb929c07 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.657848 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.658749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.659097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.659435 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.660219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.662255 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.685785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2750] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:18.690686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.702015 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d088e-42bf-4ddd-8f13-3fecd72bdc91 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.707866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.708854 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.709249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.709676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.710516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.712255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.724949 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2751] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:18.729335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.740170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529d7c6-5cf3-4ce6-942d-a733ec9a4207 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.745652 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.746350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.746640 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.746921 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.747667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.749342 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2befe-0349-43c9-baf2-ada3bc0fc19b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.761567 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2752] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:18.764595 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:18.776192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72d96-01de-431f-91d3-9d0a9311db07 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:18.780945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:18.781947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.782916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.783202 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] PATH_INFO: `/v3/projects/fd15557a0c31413dab7a502128e233e4/users/f0226f112fd549e3a7b1603649a7c69b/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.800499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd15557a0c31413dab7a502128e233e4, user_id=f0226f112fd549e3a7b1603649a7c69b, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:18.803284 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:18.818106 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2753] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:18 2022] PUT /identity/v3/projects/fd15557a0c31413dab7a502128e233e4/users/f0226f112fd549e3a7b1603649a7c69b/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:18.828305 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:18.828677 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:18.829228 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:18.857807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] MFA Rules not processed for user `f0226f112fd549e3a7b1603649a7c69b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:18.859044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:18.937283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2754] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:18.939363 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.007222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a99057-6dec-471e-9909-c95f1d177f73 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.012267 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.013038 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.013391 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.013738 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.014522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.017706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.049274 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2755] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:18 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:19.050705 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.062507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bc86b-b1ee-4ddb-9ee7-d3e4153ff411 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.095336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:19.126336 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.127065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.127351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.127627 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.128335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.130358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.160856 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2756] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:19 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:19.162423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.173628 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025332db-44d9-4953-bf9a-a9a8c500b621 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.178382 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.179063 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.179403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.179719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.180437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.182076 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.205078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2757] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:19.209377 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.220491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb60ab-4d4b-41a1-9cb0-4379d8d61506 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.230628 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.231523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.231807 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.232083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] PATH_INFO: `/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a/users/f027962b3b3f4899a83ff09a8a0ae9d0/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.253267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46baf1b5ce6f45dbad51cc97a336a43a, user_id=f027962b3b3f4899a83ff09a8a0ae9d0, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.256039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.274729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2758] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:19 2022] PUT /identity/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a/users/f027962b3b3f4899a83ff09a8a0ae9d0/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:19.278697 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.291264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821aa3d6-c1be-4187-9b12-4448d28c0123 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.376402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:19.387013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.387799 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.388264 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.389010 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.390043 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.392278 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b08d9-8e0a-4107-b7be-7fb726659299 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.406177 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2759] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:19.410555 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.423285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86613495-c7e8-4930-931d-f079b7f02e43 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.429040 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.430226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.430586 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.446782 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] PATH_INFO: `/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a/users/f027962b3b3f4899a83ff09a8a0ae9d0/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.453126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46baf1b5ce6f45dbad51cc97a336a43a, user_id=f027962b3b3f4899a83ff09a8a0ae9d0, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.456162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.473948 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2760] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:19 2022] PUT /identity/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a/users/f027962b3b3f4899a83ff09a8a0ae9d0/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:19.485888 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.488365 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.488690 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.523046 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] MFA Rules not processed for user `f027962b3b3f4899a83ff09a8a0ae9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:19.524251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:19.605678 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2761] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:19.618332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:19.620143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:19.714844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3a334-ceb1-4169-9dfb-61f487273a05 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:19.721686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:19.721686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:19.721686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:19.721686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:19.731776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:19.733410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da168bac-4b7f-40f3-aa48-1f748f2fbe48 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:19.745122 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:21.281275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:21.350831 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad646d-95fc-45e2-935a-f9fed738343b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:21.356362 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:21.357277 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:21.357568 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:21.357847 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] PATH_INFO: `/v3/users/f0226f112fd549e3a7b1603649a7c69b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:21.359095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0226f112fd549e3a7b1603649a7c69b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:21.361306 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:21.411437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0226f112fd549e3a7b1603649a7c69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:21.412052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0226f112fd549e3a7b1603649a7c69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:21.413970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0226f112fd549e3a7b1603649a7c69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:21.433633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0226f112fd549e3a7b1603649a7c69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:21.443862 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0226f112fd549e3a7b1603649a7c69b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:21.449729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:21.458348 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2763] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:21 2022] DELETE /identity/v3/users/f0226f112fd549e3a7b1603649a7c69b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:21.463103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:21.463615 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:21.463718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:21.492350 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:21.493928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:21.562370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2764] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:21.572261 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:21.573805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:21.654116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d5b59-bad5-49b9-93b7-60eaaf5883ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:21.658338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:21.658990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:21.659280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:21.659556 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:21.669054 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:21.670425 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:21.679965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2765] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:22.046659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:22.048150 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:22.056727 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:22.061014 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c2144-fdb6-4f9e-9ec9-61ab36fb9b25 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:22.067554 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:22.068732 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.069162 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.069507 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] PATH_INFO: `/v3/projects/fd15557a0c31413dab7a502128e233e4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.071578 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd15557a0c31413dab7a502128e233e4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:22.074272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:22.124974 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd15557a0c31413dab7a502128e233e4', 'request_id': 'req-120f69e5-7e32-4e4e-988f-59ae6a1201ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.130631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd15557a0c31413dab7a502128e233e4', 'request_id': 'req-120f69e5-7e32-4e4e-988f-59ae6a1201ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.137109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd15557a0c31413dab7a502128e233e4', 'request_id': 'req-120f69e5-7e32-4e4e-988f-59ae6a1201ff'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.155947 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] The token cache is being invalidate because project fd15557a0c31413dab7a502128e233e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:22.156240 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.156951 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.158623 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2766] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:22 2022] DELETE /identity/v3/projects/fd15557a0c31413dab7a502128e233e4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:22.162892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:22.173200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48da250-0ee3-415c-a037-7f2a2a94fa0b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:22.232972 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:22.233851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.234140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.234421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] PATH_INFO: `/v3/users/f027962b3b3f4899a83ff09a8a0ae9d0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.235811 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f027962b3b3f4899a83ff09a8a0ae9d0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:22.237822 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:22.293029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f027962b3b3f4899a83ff09a8a0ae9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.293880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f027962b3b3f4899a83ff09a8a0ae9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.294784 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f027962b3b3f4899a83ff09a8a0ae9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.309018 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f027962b3b3f4899a83ff09a8a0ae9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.318754 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fb16b9b6-5952-4f11-8dca-7cd4434a57ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f027962b3b3f4899a83ff09a8a0ae9d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.327630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:22.333580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2767] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:22 2022] DELETE /identity/v3/users/f027962b3b3f4899a83ff09a8a0ae9d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:22.613190 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:22.614753 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:22.635487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f69e5-7e32-4e4e-988f-59ae6a1201ff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:22.694755 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:22.695659 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.695945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.696223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] PATH_INFO: `/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.697498 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46baf1b5ce6f45dbad51cc97a336a43a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:22.699418 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:22.748281 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46baf1b5ce6f45dbad51cc97a336a43a', 'request_id': 'req-99e7a0c3-77b3-4c58-8d70-348d6db0da38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.754984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46baf1b5ce6f45dbad51cc97a336a43a', 'request_id': 'req-99e7a0c3-77b3-4c58-8d70-348d6db0da38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.761796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46baf1b5ce6f45dbad51cc97a336a43a', 'request_id': 'req-99e7a0c3-77b3-4c58-8d70-348d6db0da38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.777083 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] The token cache is being invalidate because project 46baf1b5ce6f45dbad51cc97a336a43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:22.777367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.778059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-99e7a0c3-77b3-4c58-8d70-348d6db0da38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:22.779679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2768] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:22 2022] DELETE /identity/v3/projects/46baf1b5ce6f45dbad51cc97a336a43a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:22.784821 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.785096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.785371 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.826389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:22.827616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:22.893014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2769] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:22.901797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.902069 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.902338 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.925819 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:22.927043 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:22.936390 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2770] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:22.940898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:22.950737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c4ab1-fe27-4c71-9e9d-edf2c3d46650 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:22.955477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:22.956136 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.956438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.956793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.957557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:22.957865 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:22.959459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:22.971199 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2771] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:22.975406 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:22.985065 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3840f-a89d-461a-8840-5e4a7df6fc32 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:22.989870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:22.990459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:22.990744 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:22.991018 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:22.991723 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:22.994000 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.003181 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:23.025919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2772] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:23.029947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.042427 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d927b-ee12-4422-93c0-c743250445e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.116630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.117558 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.118346 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.118776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.120309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.122851 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.146544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2773] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:23 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:23.152919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.171927 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7b283-420a-4a21-b03b-bdee76967c8e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.177317 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.178106 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.178463 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.178812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.179624 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.181574 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.199061 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2774] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:23.205130 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.215448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1e00d-93e3-43a6-96e2-d7af29baf965 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.220761 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.221431 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.221723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.222007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.222775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.224381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1754429-1530-42d6-ba17-eb0b9a4d3e2c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.234905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2775] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:23.238998 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.249332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1b6bb-cd99-45bd-9558-e80928b114e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.254992 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.256015 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.256340 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.256688 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] PATH_INFO: `/v3/projects/322f5a8de9274586a559ceaea36f7438/users/7692478dd71043d5a4dad9f1eed7f3fe/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.272619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322f5a8de9274586a559ceaea36f7438, user_id=7692478dd71043d5a4dad9f1eed7f3fe, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.275380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.289287 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2776] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:23 2022] PUT /identity/v3/projects/322f5a8de9274586a559ceaea36f7438/users/7692478dd71043d5a4dad9f1eed7f3fe/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:23.299468 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.300047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.300047 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.326775 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] MFA Rules not processed for user `7692478dd71043d5a4dad9f1eed7f3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:23.328007 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:23.367540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:23.395794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2777] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:23.400262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.463126 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dffaf-04ca-4b62-b651-9252674a6a88 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.467959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.468757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.469284 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.469480 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.470174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.472131 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.502052 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2778] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:23 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:23.503655 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.515629 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8388c-c146-4659-98c5-f0ca266dde3b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.574637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.575638 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.575898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.576247 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.577129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.579235 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.603824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2779] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:23 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:23.605042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.616581 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d33cea-e7d9-4912-a958-ab9c1b005e7a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.623089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.623803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.624085 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.624360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.625124 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.626859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.638307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2780] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:23.642793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.653265 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a7d4b-0825-40c4-9547-79049ff604de admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.658304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.659204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.659487 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.659762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] PATH_INFO: `/v3/projects/047a798e56ae4dc684b8dfb9d28ca487/users/3ddc2ffdaf8146fe92299c9a7857d560/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.683122 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047a798e56ae4dc684b8dfb9d28ca487, user_id=3ddc2ffdaf8146fe92299c9a7857d560, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.686035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.701980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2781] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:23 2022] PUT /identity/v3/projects/047a798e56ae4dc684b8dfb9d28ca487/users/3ddc2ffdaf8146fe92299c9a7857d560/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:23.705964 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.716544 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77325adc-5415-4445-a50b-bcb9c7a0d26c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.777128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.777809 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.778096 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.778368 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.779104 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.780743 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98265344-9d3a-4c80-a814-e09b6dac5533 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.795400 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2782] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:23.796620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.807236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9428789-bdb0-44a3-9e63-731828135a09 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:23.812125 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:23.813101 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.813381 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.813755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] PATH_INFO: `/v3/projects/047a798e56ae4dc684b8dfb9d28ca487/users/3ddc2ffdaf8146fe92299c9a7857d560/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.821677 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047a798e56ae4dc684b8dfb9d28ca487, user_id=3ddc2ffdaf8146fe92299c9a7857d560, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:23.824525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:23.835864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2783] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:23 2022] PUT /identity/v3/projects/047a798e56ae4dc684b8dfb9d28ca487/users/3ddc2ffdaf8146fe92299c9a7857d560/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:23.849037 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:23.849393 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:23.849741 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:23.878296 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] MFA Rules not processed for user `3ddc2ffdaf8146fe92299c9a7857d560`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:23.879590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:23.891933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:23.952563 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2784] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:23.965688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:23.968224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:24.025650 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfef01-eb77-4863-bfdc-96bf9dfecb25 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:24.029642 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:24.030283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:24.030569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:24.030849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:24.040735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:24.042161 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143ab01-b5d3-40ba-b390-d14540329b99 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:24.051988 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2785] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:24.085108 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:24.168903 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd446c-64b2-4305-a018-1e603377fb8f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:24.174195 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:24.175725 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:24.176013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:24.176297 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] PATH_INFO: `/v3/users/7692478dd71043d5a4dad9f1eed7f3fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:24.177950 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7692478dd71043d5a4dad9f1eed7f3fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:24.179906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:24.238968 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7692478dd71043d5a4dad9f1eed7f3fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:24.239849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7692478dd71043d5a4dad9f1eed7f3fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:24.257944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7692478dd71043d5a4dad9f1eed7f3fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:24.258716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7692478dd71043d5a4dad9f1eed7f3fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:24.266535 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7692478dd71043d5a4dad9f1eed7f3fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:24.282665 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2786] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:24 2022] DELETE /identity/v3/users/7692478dd71043d5a4dad9f1eed7f3fe => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:24.287172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:24.287465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:24.287746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:24.308776 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:24.309999 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:24.342334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:24.383614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2787] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:24.394233 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:24.395679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:24.428214 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:24.457742 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cfca8-542d-4ddd-b526-54c5a53d77c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:24.462360 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:24.463017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:24.463307 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:24.463590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:24.473758 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:24.475213 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:24.488556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2788] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:25.180624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:25.182241 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:25.195385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2f971-b19a-4975-92a3-0eddeb390d98 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:25.200028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:25.201016 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:25.201361 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:25.201699 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] PATH_INFO: `/v3/projects/322f5a8de9274586a559ceaea36f7438` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:25.203892 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322f5a8de9274586a559ceaea36f7438)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:25.206200 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:25.271156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322f5a8de9274586a559ceaea36f7438', 'request_id': 'req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.280268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322f5a8de9274586a559ceaea36f7438', 'request_id': 'req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.291647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322f5a8de9274586a559ceaea36f7438', 'request_id': 'req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.305154 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] The token cache is being invalidate because project 322f5a8de9274586a559ceaea36f7438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:25.305525 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.306251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.307889 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2789] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:25 2022] DELETE /identity/v3/projects/322f5a8de9274586a559ceaea36f7438 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:25.311880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:25.322261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9e67-24b0-40ed-878b-83792d756093 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:25.387385 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:25.388263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:25.388938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:25.388938 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] PATH_INFO: `/v3/users/3ddc2ffdaf8146fe92299c9a7857d560` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:25.390074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ddc2ffdaf8146fe92299c9a7857d560)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:25.392121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:25.452949 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ddc2ffdaf8146fe92299c9a7857d560'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.453880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ddc2ffdaf8146fe92299c9a7857d560'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.468452 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ddc2ffdaf8146fe92299c9a7857d560'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.469880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ddc2ffdaf8146fe92299c9a7857d560'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.478057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9715777-c19e-445d-9604-02df00a00c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ddc2ffdaf8146fe92299c9a7857d560'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:25.495983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2790] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:25 2022] DELETE /identity/v3/users/3ddc2ffdaf8146fe92299c9a7857d560 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:25.997262 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:25.998729 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:26.012779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fa01a-13b2-4fd2-bf6a-9af9c29f7975 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.096304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.097222 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.097504 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.097780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] PATH_INFO: `/v3/projects/047a798e56ae4dc684b8dfb9d28ca487` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.099028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=047a798e56ae4dc684b8dfb9d28ca487)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.101094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.149491 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047a798e56ae4dc684b8dfb9d28ca487', 'request_id': 'req-83ad6769-0892-4734-875b-4fb7d11cbc95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:26.155170 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047a798e56ae4dc684b8dfb9d28ca487', 'request_id': 'req-83ad6769-0892-4734-875b-4fb7d11cbc95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:26.156959 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:26.173877 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047a798e56ae4dc684b8dfb9d28ca487', 'request_id': 'req-83ad6769-0892-4734-875b-4fb7d11cbc95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:26.188020 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] The token cache is being invalidate because project 047a798e56ae4dc684b8dfb9d28ca487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:26.188692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:26.189536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ad6769-0892-4734-875b-4fb7d11cbc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:26.191791 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2791] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:25 2022] DELETE /identity/v3/projects/047a798e56ae4dc684b8dfb9d28ca487 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:26.196865 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.197374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.197868 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.219302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:26.220505 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:26.277511 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:26.296569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2792] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:26.304195 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.304563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.304846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.323441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:26.324691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:26.333069 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2793] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:26.337521 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.347654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef70e19-8f00-4594-be28-46dfc355fd22 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.352387 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.353071 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.353353 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.353626 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.354349 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:26.354655 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.356256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.368175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2794] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 03:21:26.372488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.382557 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2fbfc-bd4d-4342-b436-e95a63eddf2d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.387665 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.388319 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.388630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.388908 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.389618 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.391536 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.419501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2795] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:26.423777 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.434060 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900bad4-3c5c-47bf-9de2-2143d5ac38c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.493760 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.494431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.494718 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.495004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.495694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.497667 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.521331 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2796] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:26.525634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.536596 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91712-e285-4b9d-a9d1-cafdd9525896 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.541567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.542224 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.542506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.542780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.543507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.545128 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.556129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2797] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:26.560175 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.571322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e58259-4b16-4056-b6ab-d011212e83be admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.577596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.578272 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.578587 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.578860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.579603 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.581436 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5ba2-a7aa-44cd-be90-65e4e2e85e50 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.593103 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2798] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:26.597734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.608394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe31a-4e36-4360-893e-e1b0f1ef768f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.614465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.615351 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.615645 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.615916 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] PATH_INFO: `/v3/projects/108c372c474e428b9a4d9f82ad969837/users/791e077dfadb425d9151767c7d558b43/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.632345 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108c372c474e428b9a4d9f82ad969837, user_id=791e077dfadb425d9151767c7d558b43, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.635028 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.651299 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2799] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:26 2022] PUT /identity/v3/projects/108c372c474e428b9a4d9f82ad969837/users/791e077dfadb425d9151767c7d558b43/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:26.661804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.662097 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.662374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.690243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] MFA Rules not processed for user `791e077dfadb425d9151767c7d558b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:26.691470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:26.766143 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2800] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:26.770404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.849716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4a131-af9e-468a-9a11-773aec8c3ed7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.855725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.856402 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.856723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.856994 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.857832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.859667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.889196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2801] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:26.893945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:26.904679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee0711-ea5e-4ff5-ad92-c98c275fd2cf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:26.949658 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:26.963721 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:26.964402 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:26.964726 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:26.965004 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:26.965716 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:26.967660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:26.990827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2802] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:26 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:26.995542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:27.006077 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0c661-dd19-4b2c-8286-8413ec279038 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:27.011765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:27.012446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.012805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.013052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.013780 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:27.015394 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:27.026507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2803] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:27.030759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:27.041035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e641e-2c6f-47c7-9e83-dc935495c75a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:27.046024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:27.046906 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.047455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.048074 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] PATH_INFO: `/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa/users/d945ed10973d419d8b4d4b543167ba0e/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.063616 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3f6f327f164a6ba7c6874235fb21fa, user_id=d945ed10973d419d8b4d4b543167ba0e, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:27.066330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:27.081372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2804] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:27 2022] PUT /identity/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa/users/d945ed10973d419d8b4d4b543167ba0e/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:27.086635 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:27.098230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193837fe-0b89-4b17-b340-334ab4da9d04 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:27.158400 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:27.165739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:27.166494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.167606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.167606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.168075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:27.169895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43c61f-c629-42e9-95af-ee763dfe7ca0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:27.180334 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2805] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:27.184649 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:27.195177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4f1a-548c-40b6-a2f7-364360a5c5f1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:27.199959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:27.200885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.201166 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.201445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] PATH_INFO: `/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa/users/d945ed10973d419d8b4d4b543167ba0e/roles/60c47ed962914bf8985b2ab9f7d94f07` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.203483 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3f6f327f164a6ba7c6874235fb21fa, user_id=d945ed10973d419d8b4d4b543167ba0e, role_id=60c47ed962914bf8985b2ab9f7d94f07)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:27.206209 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:27.216999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2806] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:27 2022] PUT /identity/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa/users/d945ed10973d419d8b4d4b543167ba0e/roles/60c47ed962914bf8985b2ab9f7d94f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:27.228249 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.228575 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.228859 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.257000 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] MFA Rules not processed for user `d945ed10973d419d8b4d4b543167ba0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:27.258244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:27.329597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2807] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:27.340673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:27.342594 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:27.404024 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6222abc-9438-4dd4-8409-4a08df527811 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:27.408146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:27.408813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:27.409094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:27.409366 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:27.418980 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:27.420484 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:27.430262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:28.103158 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:28.199810 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66ce5-acbf-4c3f-bd29-6ed704eb404f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:28.207285 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:28.208192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:28.209260 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:28.210219 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] PATH_INFO: `/v3/users/791e077dfadb425d9151767c7d558b43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:28.211529 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791e077dfadb425d9151767c7d558b43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:28.214311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:28.278896 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791e077dfadb425d9151767c7d558b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:28.279565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791e077dfadb425d9151767c7d558b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:28.280245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791e077dfadb425d9151767c7d558b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:28.304187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791e077dfadb425d9151767c7d558b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:28.316570 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-784793ec-19c8-4a53-acc0-8a767d38efe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791e077dfadb425d9151767c7d558b43'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:28.337116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:28.337370 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:28.341109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:28.341688 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2809] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:28 2022] DELETE /identity/v3/users/791e077dfadb425d9151767c7d558b43 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:28.347826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:28.348576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:28.349089 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:28.380567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:28.381824 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:28.496446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2810] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:28.504815 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef305-a2b8-425f-a2ff-368ac76dd7f0 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:28.509250 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:28.510880 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:28.558696 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:28.596736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:28.597420 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:28.597711 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:28.597991 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:28.599882 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ee8c5-56f5-426b-84ae-c0b881f3ee68 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:28.603736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:28.604404 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:28.604719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:28.604998 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:28.605792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:28.618663 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:28.620137 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:28.620891 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:28.622612 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:28.635438 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2811] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 03:21:28.643938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2812] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:28.998759 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.000260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:29.014569 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c715-66ba-4dcc-9ee1-422964aa3a28 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:29.020185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:29.021219 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.021504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.021786 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] PATH_INFO: `/v3/projects/108c372c474e428b9a4d9f82ad969837` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.023140 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108c372c474e428b9a4d9f82ad969837)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:29.025144 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:29.077223 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108c372c474e428b9a4d9f82ad969837', 'request_id': 'req-68e788c3-09f8-4fef-8a94-5d40b35007a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.083280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108c372c474e428b9a4d9f82ad969837', 'request_id': 'req-68e788c3-09f8-4fef-8a94-5d40b35007a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.090021 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108c372c474e428b9a4d9f82ad969837', 'request_id': 'req-68e788c3-09f8-4fef-8a94-5d40b35007a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.104177 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] The token cache is being invalidate because project 108c372c474e428b9a4d9f82ad969837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:29.104493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.105134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.106691 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2813] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:28 2022] DELETE /identity/v3/projects/108c372c474e428b9a4d9f82ad969837 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:29.112082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.122516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54304a-9afb-4a36-b68c-7ffeb957a131 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:29.180244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:29.181155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.181435 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.181710 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] PATH_INFO: `/v3/users/d945ed10973d419d8b4d4b543167ba0e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.182938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d945ed10973d419d8b4d4b543167ba0e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:29.184879 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:29.231276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:29.233989 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd945ed10973d419d8b4d4b543167ba0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.235073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd945ed10973d419d8b4d4b543167ba0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.235815 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd945ed10973d419d8b4d4b543167ba0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.248900 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd945ed10973d419d8b4d4b543167ba0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.257096 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3b86ee11-b855-4b46-b274-3d63ca500447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd945ed10973d419d8b4d4b543167ba0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.269653 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2814] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:29 2022] DELETE /identity/v3/users/d945ed10973d419d8b4d4b543167ba0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:29.540710 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.542156 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:29.555235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e788c3-09f8-4fef-8a94-5d40b35007a6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:29.615653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:29.616680 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.616993 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.617265 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] PATH_INFO: `/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.618862 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3f6f327f164a6ba7c6874235fb21fa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:29.620934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:29.677089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3f6f327f164a6ba7c6874235fb21fa', 'request_id': 'req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.678660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:29.686332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3f6f327f164a6ba7c6874235fb21fa', 'request_id': 'req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.694079 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3f6f327f164a6ba7c6874235fb21fa', 'request_id': 'req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.711712 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] The token cache is being invalidate because project bb3f6f327f164a6ba7c6874235fb21fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:29.712239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.713749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee8c4e41-a51d-4af0-8d71-2686a5583aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:29.715465 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2815] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:29 2022] DELETE /identity/v3/projects/bb3f6f327f164a6ba7c6874235fb21fa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:29.720403 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.720719 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.721003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.739849 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:29.741329 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:29.821952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2816] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:29.829672 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.830058 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.830431 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.852833 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:29.854537 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:29.864462 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2817] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:29.867734 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.878259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef6ab-8967-4135-8f8e-0ef8907a0ffe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:29.883045 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:29.883717 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.884003 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.884280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.885053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:29.885359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:29.886983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:29.899524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2818] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:29.903639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.915123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc48ff3-a99c-4e4b-ad55-8e8f24ae7c85 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:29.920681 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:29.921453 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:29.921854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:29.922346 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:29.923165 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:29.925256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:29.954246 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2819] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:29 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:29.958360 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:29.988153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f110f4d-2d9d-49af-9ff6-1303e415cce2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:30.048762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:30.049444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.049733 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.050013 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.050709 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:30.052790 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:30.075421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2820] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:29 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:30.081584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:30.094394 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9be6-360e-44b4-bbb0-7eeb0fbfef7f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:30.099869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:30.100593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.100869 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.101142 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.101880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:30.103506 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:30.115089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2821] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:30.119501 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:30.130374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbafcea-e778-4fde-8520-120b1b99b0af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:30.135609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:30.136289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.136611 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.136898 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.137633 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:30.139244 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d07b1-5a9d-44b7-a592-0ebed71d1ee7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:30.157779 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2822] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:30.161893 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:30.173384 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300b7b4-794a-493d-a690-1795bdcc5c29 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:30.178827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:30.179733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.180020 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.180299 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] PATH_INFO: `/v3/projects/676167b777cb4a92b1d3e4dcf6054413/users/f54f4c4b76e645429ef271aa086bec13/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.197057 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=676167b777cb4a92b1d3e4dcf6054413, user_id=f54f4c4b76e645429ef271aa086bec13, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:30.199826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:30.215178 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2823] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:30 2022] PUT /identity/v3/projects/676167b777cb4a92b1d3e4dcf6054413/users/f54f4c4b76e645429ef271aa086bec13/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:30.226379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.226735 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.227039 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.257612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] MFA Rules not processed for user `f54f4c4b76e645429ef271aa086bec13`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:30.259141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:30.278572 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:30.344742 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2824] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:30.355770 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:30.357638 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:30.420389 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb308df-4257-4494-bd6c-c9cbaf010d4a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:30.424116 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:30.424795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:30.425077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:30.425352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:30.433504 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:30.434944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:30.443729 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:31.738816 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:31.740292 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:31.839494 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68453819-f836-4ea0-b714-c2f3166d7460 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:31.847515 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c16671fd5844a77ac6e56dff36f86f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:31.848052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:31.848516 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:31.848974 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:31.851518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:31.871050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:31.873379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:31.887004 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2826] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:37.847280 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:37.849514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:37.916870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462338a1-bdce-4af4-a2fa-4107f8b2094f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:37.921969 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:37.923033 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:37.923354 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:37.923826 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:37.926813 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:37.977127 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:38.044923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:38.046411 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41475d4f-f453-45d5-8987-071140131530 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:38.066374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:40.150433 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:40.222826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e8ea8-fec5-457f-af32-10a26bbcbf18 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:40.227684 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:40.228675 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:40.228963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:40.229257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] PATH_INFO: `/v3/users/f54f4c4b76e645429ef271aa086bec13` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:40.230453 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54f4c4b76e645429ef271aa086bec13)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:40.232384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:40.278878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:40.281455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54f4c4b76e645429ef271aa086bec13'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:40.282155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54f4c4b76e645429ef271aa086bec13'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:40.282914 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54f4c4b76e645429ef271aa086bec13'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:40.296127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54f4c4b76e645429ef271aa086bec13'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:40.304723 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54f4c4b76e645429ef271aa086bec13'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:40.317656 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2828] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:40 2022] DELETE /identity/v3/users/f54f4c4b76e645429ef271aa086bec13 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:40.322370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:40.322870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:40.322987 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:40.341514 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:40.342860 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:40.407932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2829] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:40.417875 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:40.419458 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:40.482310 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d9161-baf7-4cbd-9732-7f5b33434c10 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:40.486493 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:40.487155 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:40.487441 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:40.487725 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:40.498112 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:40.499981 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc894a5-4201-4752-b311-d86c4dede28e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:40.510647 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2830] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:40.904007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:40.905812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:40.938910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e324db-e200-4ed7-bf9f-32a2242d1721 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:40.956372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:40.958171 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:40.958796 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:40.959202 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] PATH_INFO: `/v3/projects/676167b777cb4a92b1d3e4dcf6054413` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:40.962039 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=676167b777cb4a92b1d3e4dcf6054413)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:40.964319 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.020976 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '676167b777cb4a92b1d3e4dcf6054413', 'request_id': 'req-74675e6c-b99b-401e-8695-304fcd87a307'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:41.026596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '676167b777cb4a92b1d3e4dcf6054413', 'request_id': 'req-74675e6c-b99b-401e-8695-304fcd87a307'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:41.033785 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '676167b777cb4a92b1d3e4dcf6054413', 'request_id': 'req-74675e6c-b99b-401e-8695-304fcd87a307'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:41.049111 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] The token cache is being invalidate because project 676167b777cb4a92b1d3e4dcf6054413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:41.049423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:41.050151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-74675e6c-b99b-401e-8695-304fcd87a307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:41.052276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2831] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:40 2022] DELETE /identity/v3/projects/676167b777cb4a92b1d3e4dcf6054413 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:41.056971 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.057408 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.057846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.077048 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:41.078396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:41.149388 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2832] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:41.151546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.151846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.152134 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.171881 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:41.173367 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:41.181653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2833] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:41.185615 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.196668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb288da-f3ba-4b45-a861-649614ea38de None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.202037 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.202867 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.203156 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.203593 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.204331 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:41.204668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.206259 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.223926 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2834] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:41.226803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.237573 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ad6a4-d06a-48bc-9fb0-7e97fd0eb342 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.242493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.243536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.243880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.244218 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.245042 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.247027 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.281734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:41.295805 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2835] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:41 2022] POST /identity/v3/projects => generated 359 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:41.297304 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.308237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa52fdb-40dc-41b9-9429-fd9de0d5bed1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.378354 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:41.390366 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.390607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.390736 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.390980 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.391691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.394080 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.417752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2836] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:41 2022] POST /identity/v3/users => generated 280 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:41.422459 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.435322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e688-7d2b-4a9e-b496-94068f67690a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.440800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.441472 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.441754 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.442034 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.442766 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.444381 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.467266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2837] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:41.468275 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.480846 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4482e97-26f1-4eed-b9dd-2055fb24bad9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.487755 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.488607 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.488958 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.489300 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.490455 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.493167 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091488a-548f-438e-9ec0-4b9770712458 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.505794 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2838] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:41.510540 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.521800 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d45df5-56c3-4f11-9f8a-2a3ba10f8ece admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.527072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.528121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.529221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.529221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] PATH_INFO: `/v3/projects/03a792870c4c466e8de011aa279be0ab/users/daeb4df70a8743bba59af8f5392661b2/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.545990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a792870c4c466e8de011aa279be0ab, user_id=daeb4df70a8743bba59af8f5392661b2, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.548776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.569564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2839] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:41 2022] PUT /identity/v3/projects/03a792870c4c466e8de011aa279be0ab/users/daeb4df70a8743bba59af8f5392661b2/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:41.579806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.580180 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.580630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.615978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] MFA Rules not processed for user `daeb4df70a8743bba59af8f5392661b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:41.618405 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:41.714970 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2840] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:41 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:21:41.750556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:41.753885 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:41.857380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddd8e4-27f8-420e-b1b6-def7c425124e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:41.862686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:41.863527 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:41.863983 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:41.864957 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:41.875618 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:41.877546 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3759c-1116-4d28-a1df-489907626222 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:41.890046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2841] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:41 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:41.904032 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:42.048066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5612c7-634a-44e3-9b79-d25787f7f17f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:42.053030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:42.054313 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.054706 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.055082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] PATH_INFO: `/v3/users/daeb4df70a8743bba59af8f5392661b2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.056446 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daeb4df70a8743bba59af8f5392661b2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:42.058711 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:42.130086 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daeb4df70a8743bba59af8f5392661b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.131187 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daeb4df70a8743bba59af8f5392661b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.132612 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daeb4df70a8743bba59af8f5392661b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.149349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:42.151966 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daeb4df70a8743bba59af8f5392661b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.166252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daeb4df70a8743bba59af8f5392661b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.185855 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2842] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:41 2022] DELETE /identity/v3/users/daeb4df70a8743bba59af8f5392661b2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:42.191523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.191880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.192225 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.209846 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:42.211180 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:42.273035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2843] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:42.284233 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:42.285819 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:42.360212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58b70-ce1f-4529-8239-786c3ddff809 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:42.364254 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:42.365041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.365459 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.365812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.377143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:42.378606 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fb71-4ad5-4737-8085-30bb7c7a12c3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:42.389165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2844] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:42.668013 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:42.669654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:42.685359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55881bcb-5e00-4555-9ede-45e60b2e7cd1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:42.690250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:42.691161 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.691458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.691731 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] PATH_INFO: `/v3/projects/03a792870c4c466e8de011aa279be0ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.694054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a792870c4c466e8de011aa279be0ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:42.696458 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:42.746662 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a792870c4c466e8de011aa279be0ab', 'request_id': 'req-5ee51534-ec93-434a-a9e1-05b130eea6fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.752235 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a792870c4c466e8de011aa279be0ab', 'request_id': 'req-5ee51534-ec93-434a-a9e1-05b130eea6fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.754679 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:42.759186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a792870c4c466e8de011aa279be0ab', 'request_id': 'req-5ee51534-ec93-434a-a9e1-05b130eea6fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.773158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] The token cache is being invalidate because project 03a792870c4c466e8de011aa279be0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:42.773530 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.774077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ee51534-ec93-434a-a9e1-05b130eea6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:42.775656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2845] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:42 2022] DELETE /identity/v3/projects/03a792870c4c466e8de011aa279be0ab => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:42.780964 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25205170-bae8-4edc-9463-a206a524524c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.781283 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25205170-bae8-4edc-9463-a206a524524c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.781659 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25205170-bae8-4edc-9463-a206a524524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.800600 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-25205170-bae8-4edc-9463-a206a524524c None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:42.801840 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25205170-bae8-4edc-9463-a206a524524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:42.862580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2846] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:42.870880 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.871176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.871464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.891423 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:42.892654 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:42.901417 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2847] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:42.906237 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25205170-bae8-4edc-9463-a206a524524c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:42.918276 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25205170-bae8-4edc-9463-a206a524524c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:42.923573 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:42.924236 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.924559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.924845 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.925667 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:42.926337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:42.927986 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:42.941952 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2848] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:42.948017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:42.958656 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d9faa-ae57-4b5a-804b-27e8c7b563aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:42.964169 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:42.964892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:42.965179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:42.965456 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:42.966173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:42.968123 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:42.999372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2849] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:42 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:43.002411 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:43.019061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9709304-aacb-490e-bc7c-45f56f323505 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:43.080691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:43.081465 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.081858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.082190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.082965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:43.085085 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:43.099787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:43.110428 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2850] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:43 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:43.111872 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:43.122945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff9f7b-0b11-4ca7-8b4b-601eb0d50a08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:43.128804 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:43.129570 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.129918 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.130260 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.131046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:43.132763 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:43.143609 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2851] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:43.147841 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:43.162129 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e04905-2421-4bed-996f-8ece622f2cb0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:43.167337 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:43.168082 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.168444 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.168885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.169708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:43.171782 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d560-7b65-4ce3-a6ba-969caf25f232 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:43.183983 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2852] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:43.189335 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:43.204173 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e4e09-2733-4b71-a292-0e7ce0586537 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:43.210199 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:43.211397 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.211764 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.212112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] PATH_INFO: `/v3/projects/2af2abe34f2e477a9aa23fe24051a811/users/39f3e413f5c4405aa7a010290115854e/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.232337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af2abe34f2e477a9aa23fe24051a811, user_id=39f3e413f5c4405aa7a010290115854e, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:43.235372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:43.251493 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2853] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:43 2022] PUT /identity/v3/projects/2af2abe34f2e477a9aa23fe24051a811/users/39f3e413f5c4405aa7a010290115854e/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:43.319068 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.319634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.319634 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.349637 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] MFA Rules not processed for user `39f3e413f5c4405aa7a010290115854e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:43.350853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:43.418282 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2854] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:43.429932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:43.431790 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:43.494989 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b36c5-f6d4-4d81-ae72-3caee58bbff5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:43.502038 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:43.502926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:43.503408 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:43.503827 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:43.517006 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:43.518470 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:43.527877 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2855] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:44.575221 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:44.576694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:44.635664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935c9c-0242-4585-b02b-51f7134723c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:44.639630 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:44.640450 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:44.640739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:44.640990 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:44.642976 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:44.708748 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:44.710116 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:44.720060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2856] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:50.794717 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:50.796733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:50.809437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576824-a81f-4bef-981e-59835259ecc0 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:50.813737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:50.814386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:50.814669 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:50.814945 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:50.816941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:50.884839 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:50.886298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:50.897264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2857] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:53.577333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:53.585167 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:53.589969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd65b-15ed-4c76-9106-dfa52d978f42 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:53.593844 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:53.594842 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:53.595135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:53.595421 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:53.597744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:53.613032 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:53.643646 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:53.707919 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35cea4-cc80-4051-b3e0-ccb8a030752f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:53.724396 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:53.725733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:53.725733 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:53.725902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] PATH_INFO: `/v3/users/39f3e413f5c4405aa7a010290115854e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:53.727249 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f3e413f5c4405aa7a010290115854e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:53.729328 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:53.732383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:53.733181 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a9079-7bdd-49a3-8c0c-e14ec6674fe1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:53.752982 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:53.795158 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f3e413f5c4405aa7a010290115854e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:53.795883 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f3e413f5c4405aa7a010290115854e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:53.811149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f3e413f5c4405aa7a010290115854e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:53.812383 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f3e413f5c4405aa7a010290115854e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:53.821380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f3e413f5c4405aa7a010290115854e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:53.837414 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2859] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:21:53 2022] DELETE /identity/v3/users/39f3e413f5c4405aa7a010290115854e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:53.842049 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:53.842342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:53.842620 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:53.860923 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:53.862143 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:53.932563 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2860] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:53.950737 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:53.952198 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:54.020185 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb5d-02a4-4598-8f59-0e2585d5d7ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.026263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.027551 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.027988 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.028266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.042078 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.043952 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91089833-089d-40ab-a64e-88dc65f7efcc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.057913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:54.491993 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.494333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:54.511912 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41749ebb-82c1-4102-a91c-26337cbdce66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.518539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.519522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.519712 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.520271 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] PATH_INFO: `/v3/projects/2af2abe34f2e477a9aa23fe24051a811` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.521746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af2abe34f2e477a9aa23fe24051a811)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.523685 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.573829 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af2abe34f2e477a9aa23fe24051a811', 'request_id': 'req-64f09549-9699-4627-8adf-c8adf635b3d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:54.581522 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af2abe34f2e477a9aa23fe24051a811', 'request_id': 'req-64f09549-9699-4627-8adf-c8adf635b3d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:54.591050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af2abe34f2e477a9aa23fe24051a811', 'request_id': 'req-64f09549-9699-4627-8adf-c8adf635b3d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:54.602728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] The token cache is being invalidate because project 2af2abe34f2e477a9aa23fe24051a811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:21:54.603280 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:54.603932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64f09549-9699-4627-8adf-c8adf635b3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:21:54.605540 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2862] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:21:54 2022] DELETE /identity/v3/projects/2af2abe34f2e477a9aa23fe24051a811 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:21:54.610473 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.611076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.611076 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.630984 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:54.632322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:54.679149 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:54.698767 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2863] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:54.707029 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.707474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.707763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.728535 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:54.729929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:54.739030 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2864] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:54.743693 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.753643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a4de-4928-4ae9-a62a-664d556850d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.758261 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.758932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.759215 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.759491 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.760232 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 03:21:54.760577 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.762211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.775745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2865] 10.222.0.16 () {58 vars in 1152 bytes} [Tue Jun 21 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 03:21:54.779060 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.789651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301927b-9d2a-438f-be59-7de373165d8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.794580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.795357 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.795762 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.796142 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.796932 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.799253 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.826342 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2866] 10.222.0.16 () {60 vars in 1152 bytes} [Tue Jun 21 03:21:54 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 03:21:54.830795 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.841643 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a0cc5-f178-43f8-a394-7e91e3c276c4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.902523 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.903584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.903584 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.903823 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.904565 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.906613 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.931211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2867] 10.222.0.16 () {60 vars in 1143 bytes} [Tue Jun 21 03:21:54 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 03:21:54.932379 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.943969 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa9a3-24d0-454b-a30e-d38de4b59b5d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:54.951120 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:54.952035 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:54.952448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:54.952839 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:54.953765 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:54.955812 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:54.968488 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2868] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 03:21:54.980188 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:54.991788 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a8890-f320-40f5-bf5a-70a91209cebc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:55.014686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:55.016888 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:55.017445 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:55.017558 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:55.018401 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:55.020263 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d9c28-2c8f-4365-9af9-7750b9ad71be admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:55.033447 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2869] 10.222.0.16 () {58 vars in 1121 bytes} [Tue Jun 21 03:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 03:21:55.039245 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:55.047507 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:21:55.052338 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b4291-400e-4480-ab27-51b59606db04 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:55.058632 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:55.059938 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:55.060590 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:55.061252 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] PATH_INFO: `/v3/projects/f41d09ea2ec147cfafda6ee66cd063f4/users/811a703e4b684cbd9191be87e5a2cdd4/roles/de0ee47deb904e17990632bc8067f864` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:55.096073 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41d09ea2ec147cfafda6ee66cd063f4, user_id=811a703e4b684cbd9191be87e5a2cdd4, role_id=de0ee47deb904e17990632bc8067f864)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:55.098826 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:55.125773 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2870] 10.222.0.16 () {60 vars in 1482 bytes} [Tue Jun 21 03:21:55 2022] PUT /identity/v3/projects/f41d09ea2ec147cfafda6ee66cd063f4/users/811a703e4b684cbd9191be87e5a2cdd4/roles/de0ee47deb904e17990632bc8067f864 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 03:21:55.139686 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d690fa-2877-4655-a53f-5188795e402f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:55.139981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d690fa-2877-4655-a53f-5188795e402f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:55.140266 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d690fa-2877-4655-a53f-5188795e402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:55.180121 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76d690fa-2877-4655-a53f-5188795e402f None None] MFA Rules not processed for user `811a703e4b684cbd9191be87e5a2cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:55.181967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76d690fa-2877-4655-a53f-5188795e402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:55.269068 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2871] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:21:55.283651 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:21:55.286108 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:55.397694 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648186-62ee-45ee-bc25-c44312cf1418 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:21:55.402692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bc17792b4d4879a1fd533206245694', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:21:55.403771 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:55.404171 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:55.404598 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:55.419204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:21:55.420818 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:21:55.438303 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2872] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:21:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:21:56.954094 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b948-95b5-4206-9790-79b62204de66 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:21:56.955221 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b948-95b5-4206-9790-79b62204de66 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:21:56.956347 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b948-95b5-4206-9790-79b62204de66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:21:56.976866 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7127b948-95b5-4206-9790-79b62204de66 None None] MFA Rules not processed for user `811a703e4b684cbd9191be87e5a2cdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:21:56.978913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7127b948-95b5-4206-9790-79b62204de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:21:56.990598 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2873] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:21:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:01.285268 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:01.288272 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:01.386114 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b948-95b5-4206-9790-79b62204de66 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:01.388372 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7127b948-95b5-4206-9790-79b62204de66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:01.439391 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:01.466664 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf2440-cdce-4621-84b1-d9683d7b66af service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:01.475513 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:01.476731 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:01.477088 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:01.477436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:01.479385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:01.495991 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:01.498589 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:01.514196 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:22:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:22:01.517942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b948-95b5-4206-9790-79b62204de66 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:01.522910 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:01.523926 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:01.524503 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:01.524892 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] PATH_INFO: `/v3/users/41c4d3bab2164767b8eefcb4427b65b7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:01.526238 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c4d3bab2164767b8eefcb4427b65b7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:01.528776 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:01.589995 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c4d3bab2164767b8eefcb4427b65b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:01.590802 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c4d3bab2164767b8eefcb4427b65b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:01.604374 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c4d3bab2164767b8eefcb4427b65b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:01.605211 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c4d3bab2164767b8eefcb4427b65b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:01.614895 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c4d3bab2164767b8eefcb4427b65b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:01.633560 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2875] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:22:01 2022] DELETE /identity/v3/users/41c4d3bab2164767b8eefcb4427b65b7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:01.868189 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:01.888614 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cb8f0-59d3-4c49-a9a8-0fc75573fdd1 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:01.972751 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:01.986401 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:01.987297 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:01.988014 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:01.988295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] PATH_INFO: `/v3/projects/1a43462c60384cecb441ccf14de5672e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:01.989569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a43462c60384cecb441ccf14de5672e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:01.991691 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:02.042204 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a43462c60384cecb441ccf14de5672e', 'request_id': 'req-4591146c-7dc5-45ef-91af-5a9a15bdde23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.048094 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a43462c60384cecb441ccf14de5672e', 'request_id': 'req-4591146c-7dc5-45ef-91af-5a9a15bdde23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.057212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a43462c60384cecb441ccf14de5672e', 'request_id': 'req-4591146c-7dc5-45ef-91af-5a9a15bdde23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.069549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] The token cache is being invalidate because project 1a43462c60384cecb441ccf14de5672e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:22:02.069838 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.070477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.072052 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2876] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:22:01 2022] DELETE /identity/v3/projects/1a43462c60384cecb441ccf14de5672e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:02.076256 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:02.078248 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:02.092607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f080-ff25-48f3-85d9-c1be3e1ff6fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:02.152170 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:02.153146 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:02.153426 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:02.153697 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] PATH_INFO: `/v3/users/bed23ae7930b4be4b611160e42c343c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:02.171602 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed23ae7930b4be4b611160e42c343c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:02.173745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:02.224924 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:02.227639 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed23ae7930b4be4b611160e42c343c4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.228650 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed23ae7930b4be4b611160e42c343c4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.245606 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed23ae7930b4be4b611160e42c343c4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.246298 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed23ae7930b4be4b611160e42c343c4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.253357 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed23ae7930b4be4b611160e42c343c4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.268141 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2877] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:22:02 2022] DELETE /identity/v3/users/bed23ae7930b4be4b611160e42c343c4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:02.454034 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:02.455576 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:02.470124 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591146c-7dc5-45ef-91af-5a9a15bdde23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:02.532230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:02.533157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:02.533448 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:02.533728 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] PATH_INFO: `/v3/projects/755877623fbf49d78f52dac475ac83a2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:02.545866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=755877623fbf49d78f52dac475ac83a2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:02.548192 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:02.597866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755877623fbf49d78f52dac475ac83a2', 'request_id': 'req-9676c6f6-7783-4b42-97cc-92d1c1434ebe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.604654 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755877623fbf49d78f52dac475ac83a2', 'request_id': 'req-9676c6f6-7783-4b42-97cc-92d1c1434ebe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.613893 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755877623fbf49d78f52dac475ac83a2', 'request_id': 'req-9676c6f6-7783-4b42-97cc-92d1c1434ebe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.627328 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] The token cache is being invalidate because project 755877623fbf49d78f52dac475ac83a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:22:02.627621 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.628349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9676c6f6-7783-4b42-97cc-92d1c1434ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:02.630269 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2878] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:22:02 2022] DELETE /identity/v3/projects/755877623fbf49d78f52dac475ac83a2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:12.264713 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=4,keystone:SELECT=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:12.624384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:13.420304 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:13.421871 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:13.495476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93140f-97da-4d63-8818-c666c427a60d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:13.559812 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:13.560516 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:13.560808 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:13.561089 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:13.562928 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:13.575243 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:13.576900 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:13.588301 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2879] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:22:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:22:14.770752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4d57-35b7-4c59-af1c-55381839f268 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:14.771289 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4d57-35b7-4c59-af1c-55381839f268 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:14.771396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4d57-35b7-4c59-af1c-55381839f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:14.789333 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-355e4d57-35b7-4c59-af1c-55381839f268 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:22:14.790566 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-355e4d57-35b7-4c59-af1c-55381839f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:14.857623 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2880] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:14.898276 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:14.899803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:14.971327 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412581f0-98c3-422d-a8a5-3faa27f3bbff service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:14.979960 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:14.980966 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:14.981322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:14.981622 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:14.996999 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:14.999492 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:15.009262 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2881] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:22:15.542492 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5690d0-e80c-4232-976a-a1219cba057d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:15.542802 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5690d0-e80c-4232-976a-a1219cba057d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:15.543087 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5690d0-e80c-4232-976a-a1219cba057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:15.561676 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0b5690d0-e80c-4232-976a-a1219cba057d None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:22:15.562886 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b5690d0-e80c-4232-976a-a1219cba057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:15.570613 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1449/2882] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:22:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:15.583421 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:15.594302 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45d2d-8077-4290-a918-ad696575bd9a service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:15.600084 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:15.600791 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:15.601077 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:15.601352 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:15.609934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:15.611599 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:15.620226 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2883] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:22:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:22:15.833703 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0254c5-e4db-46b0-a624-1122cf1b717c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:15.834212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0254c5-e4db-46b0-a624-1122cf1b717c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:15.834212 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0254c5-e4db-46b0-a624-1122cf1b717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:15.852203 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3f0254c5-e4db-46b0-a624-1122cf1b717c None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:22:15.853420 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f0254c5-e4db-46b0-a624-1122cf1b717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:15.861763 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1450/2884] 10.222.0.16 () {58 vars in 1009 bytes} [Tue Jun 21 03:22:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:15.877067 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:15.889464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a280a5-4806-48c8-9d16-5140ad36e78b service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:15.898072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:15.898942 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:15.899541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:15.899541 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:15.908571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:15.911520 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:15.913870 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:15.922136 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2885] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:22:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:22:16.263514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893765-239e-4590-b72f-121594325641 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:16.263806 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893765-239e-4590-b72f-121594325641 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:16.264115 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893765-239e-4590-b72f-121594325641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:16.282071 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4a893765-239e-4590-b72f-121594325641 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:22:16.283309 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a893765-239e-4590-b72f-121594325641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:16.292053 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1451/2886] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:16.304749 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:16.316332 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5659d4-1cf9-41f8-8453-f39201839c53 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:16.322118 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:16.322859 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:16.323217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:16.323564 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:16.333183 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:16.334941 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661bdf3-9981-4bd9-a855-9ecac0055dbb service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:16.343783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2887] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 03:22:18.648927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a893765-239e-4590-b72f-121594325641 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:18.650752 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a893765-239e-4590-b72f-121594325641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:18.733135 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a893765-239e-4590-b72f-121594325641 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:18.738696 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:18.739592 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:18.739878 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:18.740157 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] PATH_INFO: `/v3/users/811a703e4b684cbd9191be87e5a2cdd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:18.741565 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=811a703e4b684cbd9191be87e5a2cdd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:18.744318 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:18.796257 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811a703e4b684cbd9191be87e5a2cdd4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:18.796944 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811a703e4b684cbd9191be87e5a2cdd4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:18.797539 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811a703e4b684cbd9191be87e5a2cdd4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:18.810477 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811a703e4b684cbd9191be87e5a2cdd4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:18.819808 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811a703e4b684cbd9191be87e5a2cdd4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:18.832901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1452/2888] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:22:18 2022] DELETE /identity/v3/users/811a703e4b684cbd9191be87e5a2cdd4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:18.838230 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:18.838757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:18.838757 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:18.857745 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] MFA Rules not processed for user `b424d6dcd2eb40949197b229df3c84e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:22:18.858967 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:18.920285 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2889] 10.222.0.16 () {58 vars in 957 bytes} [Tue Jun 21 03:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 03:22:18.930978 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:18.932436 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:18.939760 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:19.002737 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c79e-26dd-4960-b315-7ffe6b24de7a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:19.007569 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:19.008255 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:19.008551 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:19.008858 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:19.019474 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:19.020963 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:19.031437 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1453/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:22:19.231607 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:22:19.233109 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:22:19.246358 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11f9ec-d0a4-447c-a803-f15a863cccdc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:22:19.253476 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:22:19.254382 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:22:19.254653 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:22:19.254932 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] PATH_INFO: `/v3/projects/f41d09ea2ec147cfafda6ee66cd063f4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:22:19.256184 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41d09ea2ec147cfafda6ee66cd063f4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:22:19.258363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:22:19.317112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41d09ea2ec147cfafda6ee66cd063f4', 'request_id': 'req-52859664-13f6-41a2-9d03-0faf0200618f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:19.322920 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41d09ea2ec147cfafda6ee66cd063f4', 'request_id': 'req-52859664-13f6-41a2-9d03-0faf0200618f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:19.329251 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41d09ea2ec147cfafda6ee66cd063f4', 'request_id': 'req-52859664-13f6-41a2-9d03-0faf0200618f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:19.343619 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] The token cache is being invalidate because project f41d09ea2ec147cfafda6ee66cd063f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:22:19.343904 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:19.344547 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:22:19.346174 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2891] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:22:19 2022] DELETE /identity/v3/projects/f41d09ea2ec147cfafda6ee66cd063f4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:22:29.026901 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=24 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:22:29.340771 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:03.657386 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:03.666796 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba0b34-5469-4310-be81-674fe3f27eb3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:03.716422 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:03.717103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:03.717389 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:03.717670 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:03.719542 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:03.814383 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:03.815744 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:03.836718 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1454/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:24:05.027487 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:05.029098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:05.043902 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52859664-13f6-41a2-9d03-0faf0200618f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:05.048176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:05.048889 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:05.049179 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:05.049455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:05.051359 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:05.066322 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:05.067844 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b5429-1bd8-4d9f-97a9-e965a090bda9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:05.079253 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:24:13.833793 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:15.073934 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:18.548211 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:18.562843 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c4fd-98b3-4bbe-a704-64567c226d19 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:18.566640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:18.567374 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.567727 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.568067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.570065 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:18.580359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:18.581910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fed7a-d262-45d2-97b7-db15aff6f913 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:18.590798 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1455/2894] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:24:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:24:18.620797 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.621283 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.621395 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.682842 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] MFA Rules not processed for user `41a3f61f1038445aa7ab1224445fb2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:24:18.684168 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:18.703344 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c709-478e-44c3-90fa-ca9faa91c9ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.703616 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c709-478e-44c3-90fa-ca9faa91c9ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.703895 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8c709-478e-44c3-90fa-ca9faa91c9ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.705302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1456/2895] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:24:18.711095 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.711396 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.711708 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.758275 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2896] 10.222.0.16 () {58 vars in 967 bytes} [Tue Jun 21 03:24:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:24:18.766475 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] MFA Rules not processed for user `357dd2446e9248cab4e559dc5ce931b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:24:18.767805 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:18.775597 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1457/2897] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:24:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:24:18.787379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:18.798673 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469577cf-71be-44ce-aa4d-c747704d4e71 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:18.802702 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:18.803471 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.804370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.804370 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.812437 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:18.813164 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:18.814066 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417515b2-37bd-4105-a807-e70d522e5381 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:18.814579 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:18.823035 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:24:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:24:18.910787 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42579c-2e00-4907-a34c-c254d499af92 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:18.911250 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025d517-a2fa-4e26-94a8-5003c049290f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.911542 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025d517-a2fa-4e26-94a8-5003c049290f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.911821 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025d517-a2fa-4e26-94a8-5003c049290f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.913363 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2899] 10.222.0.16 () {54 vars in 921 bytes} [Tue Jun 21 03:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 03:24:18.916165 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b4f45c15c2949f5bd0e606b3cf00493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:18.916853 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.917232 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.917514 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.918977 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.919264 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.919538 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:18.926544 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:18.928067 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:18.936496 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1458/2900] 10.222.0.16 () {60 vars in 1384 bytes} [Tue Jun 21 03:24:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 03:24:18.958306 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] MFA Rules not processed for user `1c16671fd5844a77ac6e56dff36f86f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 03:24:18.959590 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:18.970151 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2901] 10.222.0.16 () {58 vars in 1018 bytes} [Tue Jun 21 03:24:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 03:24:18.981213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:18.991311 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb2-0792-4ea7-aa3a-868c7574256c service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:18.995814 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:18.996559 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:18.996910 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:18.997302 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:19.009024 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:19.010758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:19.022884 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1459/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:24:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:24:26.595026 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:26.596728 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:26.665112 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133e5f0-4add-4099-be58-390fcb0982af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:26.669970 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:26.670849 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:26.671128 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:26.671399 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] PATH_INFO: `/v3/users/994fd8a1975c45a8a65ace3d090d1ff5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:26.672631 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994fd8a1975c45a8a65ace3d090d1ff5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:26.674593 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:26.717267 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:26.730379 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994fd8a1975c45a8a65ace3d090d1ff5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:26.731025 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994fd8a1975c45a8a65ace3d090d1ff5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:26.742931 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994fd8a1975c45a8a65ace3d090d1ff5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:26.743595 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994fd8a1975c45a8a65ace3d090d1ff5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:26.750944 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994fd8a1975c45a8a65ace3d090d1ff5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:26.766308 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2903] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:24:26 2022] DELETE /identity/v3/users/994fd8a1975c45a8a65ace3d090d1ff5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:24:26.954574 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:26.965658 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcd0b6-dec7-4f85-898b-20caeec20b6a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:27.024023 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:27.024929 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:27.025213 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:27.025490 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] PATH_INFO: `/v3/projects/5e57658c9c9340adba6ab15050b24e8a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:27.026739 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e57658c9c9340adba6ab15050b24e8a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:27.028686 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:27.046640 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:27.082965 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e57658c9c9340adba6ab15050b24e8a', 'request_id': 'req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.089953 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e57658c9c9340adba6ab15050b24e8a', 'request_id': 'req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.100152 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e57658c9c9340adba6ab15050b24e8a', 'request_id': 'req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.113146 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] The token cache is being invalidate because project 5e57658c9c9340adba6ab15050b24e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:24:27.113438 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.114209 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.116692 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1460/2904] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:24:26 2022] DELETE /identity/v3/projects/5e57658c9c9340adba6ab15050b24e8a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:24:27.120730 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:27.122176 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:27.137519 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7705-954c-4f02-ba77-df0b19428a03 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:27.199145 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:27.200064 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:27.200351 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:27.200675 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] PATH_INFO: `/v3/users/00d3c70fd31144018c3fc93e893fa6f3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:27.201913 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d3c70fd31144018c3fc93e893fa6f3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:27.203854 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:27.253054 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d3c70fd31144018c3fc93e893fa6f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.253694 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d3c70fd31144018c3fc93e893fa6f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.265572 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d3c70fd31144018c3fc93e893fa6f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.266205 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d3c70fd31144018c3fc93e893fa6f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.273699 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d3c70fd31144018c3fc93e893fa6f3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.289589 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1445/2905] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:24:27 2022] DELETE /identity/v3/users/00d3c70fd31144018c3fc93e893fa6f3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:24:27.473624 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:24:27.475081 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:24:27.495288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9901ca-6f01-4cc2-8190-6c793dec4a5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:24:27.551873 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:24:27.552785 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:24:27.553075 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:24:27.553356 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] PATH_INFO: `/v3/projects/e182160965d24221bff6069a0b1a89a3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:24:27.554580 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e182160965d24221bff6069a0b1a89a3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:24:27.556549 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:24:27.613524 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e182160965d24221bff6069a0b1a89a3', 'request_id': 'req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.619567 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e182160965d24221bff6069a0b1a89a3', 'request_id': 'req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.627871 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e182160965d24221bff6069a0b1a89a3', 'request_id': 'req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.639103 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] The token cache is being invalidate because project e182160965d24221bff6069a0b1a89a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:24:27.639416 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.640066 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:24:27.641668 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1461/2906] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:24:27 2022] DELETE /identity/v3/projects/e182160965d24221bff6069a0b1a89a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:24:37.285736 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:24:37.636384 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:25:57.011861 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:25:57.013536 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:25:57.078072 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdfe4b-b2ae-4b11-a454-b5b3c08672d6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:25:57.130539 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:25:57.131288 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:25:57.131575 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:25:57.131853 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:25:57.133792 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:25:57.143834 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:25:57.145315 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:25:57.156343 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1446/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:25:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:25:59.311288 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:25:59.312918 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:25:59.326543 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9c3b-f38e-4650-bdf9-3eb937b1d20f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:25:59.331242 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:25:59.331905 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:25:59.332190 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:25:59.332469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:25:59.335061 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:25:59.401506 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:25:59.403693 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:25:59.415850 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1462/2908] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:26:04.225186 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:04.234455 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf66ed-9f9c-418f-acb6-40b6b4e69841 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:04.238690 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '357dd2446e9248cab4e559dc5ce931b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:04.239750 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:04.240044 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:04.240325 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:04.242556 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:04.307916 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:04.309412 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:04.320017 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1447/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:26:07.431945 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:07.436385 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:07.533141 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c581ff-a75d-4aea-8f5f-0adce9391c3f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:07.539795 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:07.540609 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:07.540758 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:07.541041 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:07.543078 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:07.555716 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:07.557349 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:07.568518 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1463/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:26:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:26:10.713972 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:10.715756 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:10.728464 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceae56-376e-48ae-9db4-ef6a6eb900d8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:10.733794 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:10.734683 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:10.734973 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:10.735241 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] PATH_INFO: `/v3/users/c3c55aee07d4447489ddedde5972ec58` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:10.736474 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c55aee07d4447489ddedde5972ec58)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:10.738410 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:10.786981 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c55aee07d4447489ddedde5972ec58'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:10.787734 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c55aee07d4447489ddedde5972ec58'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:10.788959 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:26:10.802337 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c55aee07d4447489ddedde5972ec58'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:10.803059 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c55aee07d4447489ddedde5972ec58'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:10.810376 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c55aee07d4447489ddedde5972ec58'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:10.824929 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1448/2911] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:26:10 2022] DELETE /identity/v3/users/c3c55aee07d4447489ddedde5972ec58 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:26:11.014502 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:11.026368 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613b13-e81c-4742-9ae3-2d98e6d14bf8 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:11.089497 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:11.090359 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:11.090644 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:11.090919 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] PATH_INFO: `/v3/projects/775518eb8c104022b5acfd12ead0fee5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:11.092127 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] RBAC: Authorizing `identity:delete_project(project_id=775518eb8c104022b5acfd12ead0fee5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:11.094050 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:11.132230 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:26:11.146499 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775518eb8c104022b5acfd12ead0fee5', 'request_id': 'req-fba02597-4533-46f6-a550-fc56d9a256db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.152599 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775518eb8c104022b5acfd12ead0fee5', 'request_id': 'req-fba02597-4533-46f6-a550-fc56d9a256db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.161627 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775518eb8c104022b5acfd12ead0fee5', 'request_id': 'req-fba02597-4533-46f6-a550-fc56d9a256db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.173832 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] The token cache is being invalidate because project 775518eb8c104022b5acfd12ead0fee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:26:11.174105 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.174746 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.176295 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1464/2912] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:26:11 2022] DELETE /identity/v3/projects/775518eb8c104022b5acfd12ead0fee5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:26:11.180362 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:11.182050 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:11.195803 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2f190-0617-4444-83d9-f13cd82d54fd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:11.257217 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:11.258098 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:11.258386 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:11.258660 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] PATH_INFO: `/v3/users/cd337799adad492d93eb07e245a42387` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:11.259933 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd337799adad492d93eb07e245a42387)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:11.261923 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:11.313345 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd337799adad492d93eb07e245a42387'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.313990 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd337799adad492d93eb07e245a42387'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.326380 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd337799adad492d93eb07e245a42387'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.327007 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd337799adad492d93eb07e245a42387'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.334879 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd337799adad492d93eb07e245a42387'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.350239 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1449/2913] 10.222.0.16 () {58 vars in 1223 bytes} [Tue Jun 21 03:26:11 2022] DELETE /identity/v3/users/cd337799adad492d93eb07e245a42387 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:26:11.540172 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:26:11.542469 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:26:11.558885 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba02597-4533-46f6-a550-fc56d9a256db admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:26:11.626234 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b424d6dcd2eb40949197b229df3c84e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eab06a8b789343a7bc8eebc83d2a6637', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:26:11.627140 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:26:11.627423 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:26:11.627701 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] PATH_INFO: `/v3/projects/9fe0fa323179498db3572def1af95429` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:26:11.628947 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe0fa323179498db3572def1af95429)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:26:11.630925 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:26:11.678602 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe0fa323179498db3572def1af95429', 'request_id': 'req-c5216474-5bb1-4454-bf9a-6daccdf4ed63'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.684102 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe0fa323179498db3572def1af95429', 'request_id': 'req-c5216474-5bb1-4454-bf9a-6daccdf4ed63'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.693251 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe0fa323179498db3572def1af95429', 'request_id': 'req-c5216474-5bb1-4454-bf9a-6daccdf4ed63'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.704866 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] The token cache is being invalidate because project 9fe0fa323179498db3572def1af95429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 03:26:11.705153 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.705864 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5216474-5bb1-4454-bf9a-6daccdf4ed63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 03:26:11.707505 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1465/2914] 10.222.0.16 () {58 vars in 1232 bytes} [Tue Jun 21 03:26:11 2022] DELETE /identity/v3/projects/9fe0fa323179498db3572def1af95429 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 03:26:21.346783 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=65,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:26:21.701927 dv-cinder-846842-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 03:28:01.279636 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 03:28:01.281339 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:28:01.359467 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3c80d-9789-42b6-a971-45108b68bc66 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 03:28:01.429482 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'beb0197dc69744d69c47ba294b9a4f41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '55a2591b7b2f4643b5e3599935ea51a6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 03:28:01.430046 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 03:28:01.430330 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 03:28:01.430611 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 03:28:01.432450 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 03:28:01.444661 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 03:28:01.446596 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281a65b-c8e5-49f8-b04c-3454426c68e5 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 03:28:01.458571 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1450/2915] 10.222.0.16 () {60 vars in 1365 bytes} [Tue Jun 21 03:28:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 03:28:11.454341 dv-cinder-846842-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=58 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}